Bug 1993 - Task Scheduler Needs Re-write
Summary: Task Scheduler Needs Re-write
Status: In Discussion
Alias: None
Product: Koha
Classification: Unclassified
Component: Task Scheduler (show other bugs)
Version: master
Hardware: PC All
: P3 major with 41 votes (vote)
Assignee: Galen Charlton
QA Contact: Bugs List
URL:
Keywords:
: 1554 20673 (view as bug list)
Depends on:
Blocks: 3935
  Show dependency treegraph
 
Reported: 2008-04-02 01:36 UTC by Nicole C. Engard
Modified: 2019-05-04 14:22 UTC (History)
31 users (show)

See Also:
Change sponsored?: ---
Patch complexity: Medium patch
Who signed the patch off:
Text to go in the release notes:
Version(s) released in:


Attachments
Clicked scheudle next to 'all patrons' (27.38 KB, image/png)
2008-04-02 13:38 UTC, Chris Cormack
Details
Entered in valid information (15.88 KB, image/png)
2008-04-02 13:38 UTC, Chris Cormack
Details
Clicked save and brought back to empty form (50.05 KB, image/png)
2008-04-02 13:39 UTC, Chris Cormack
Details
Proposed scheduler.pl (3.07 KB, text/plain)
2010-06-21 19:22 UTC, Jason Sherman
Details
Proposed runreport.pl (5.87 KB, text/plain)
2010-06-21 19:24 UTC, Jason Sherman
Details
Improves task scheduler (5.55 KB, patch)
2012-03-22 15:08 UTC, Sophie MEYNIEUX
Details | Diff | Splinter Review
[Signed-Off] Bug 1993: Improving task scheduler (5.61 KB, patch)
2012-04-03 14:56 UTC, Kyle M Hall
Details | Diff | Splinter Review
Bug 1993: Improving task scheduler (5.93 KB, patch)
2013-03-22 08:51 UTC, Sophie MEYNIEUX
Details | Diff | Splinter Review
Bug 1993: Improving task scheduler (5.78 KB, patch)
2013-05-07 14:40 UTC, Sophie MEYNIEUX
Details | Diff | Splinter Review
Bug 1993: Improving task scheduler (4.99 KB, patch)
2013-05-10 13:01 UTC, Sophie MEYNIEUX
Details | Diff | Splinter Review
screenshot (20.34 KB, image/png)
2013-05-17 06:44 UTC, Katrin Fischer
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Chris Cormack 2010-05-21 00:36:59 UTC


---- Reported by nengard@gmail.com 2008-04-02 13:36:44 ----

I have tried to schedule a few tasks and none work.  See my screenshots.  Basically I enter the time and the email and the date and I get no confirmation and see no list of scheduled tasks - apparently this list is supposed to show below my form.  I've tried this a few times and nothing has happened.

I also clicked on the Schedule link from the list of reports and it took me to the form with the wrong report selected in the pull down.



---- Additional Comments From nengard@gmail.com 2008-04-02 13:38:25 ----

Created an attachment
Clicked scheudle next to 'all patrons'





---- Additional Comments From nengard@gmail.com 2008-04-02 13:38:49 ----

Created an attachment
Entered in valid information





---- Additional Comments From nengard@gmail.com 2008-04-02 13:39:15 ----

Created an attachment
Clicked save and brought back to empty form





---- Additional Comments From jmf@liblime.com 2008-05-19 16:56:45 ----

Re-assigning to Galen



---- Additional Comments From gmcharlt@gmail.com 2008-06-06 10:43:46 ----

Submitted patch to bandage the major issues.  A complete rewrite of the task scheduler is needed, IMO, but time does not permit for 3.0.  At least now it should be functional for platforms that can run 'at'.



---- Additional Comments From jmf@liblime.com 2008-06-12 13:30:31 ----

OK, thanks Galen! I'm gonna mark this as an enhancement request and change the summary to 'Task Scheduler Needs Re-write'.



---- Additional Comments From nengard@gmail.com 2008-11-13 10:35:55 ----

Is this in the works for 3.2?  



---- Additional Comments From joe.atzberger@liblime.com 2009-03-20 19:09:10 ----

Please note that there are big security issues for using "at" (or in this case "atq") in a multiple-client hosting environment, since each Koha will be running as the same "apache" user.  There is no differentiation on what jobs are visible: everybody will see everything.  

Anyway, even when atq works, you can't schedule any kind of recurrence for the same job, which I suspect is what staff would want.  

A totally different implementation is probably in order, perhaps including a dispatcher daemon.



--- Bug imported by chris@bigballofwax.co.nz 2010-05-21 00:36 UTC  ---

This bug was previously known as _bug_ 1993 at http://bugs.koha.org/cgi-bin/bugzilla3/show_bug.cgi?id=1993
Imported an attachment (id=449)
Imported an attachment (id=450)
Imported an attachment (id=451)

Actual time not defined. Setting to 0.0
CC member bchurch@ptfs.com does not have an account here
CC member joe.atzberger@liblime.com does not have an account here
CC member jwagner@ptfs.com does not have an account here
The original submitter of attachment 449 [details] is unknown.
   Reassigning to the person who moved it here: chris@bigballofwax.co.nz.
The original submitter of attachment 450 [details] is unknown.
   Reassigning to the person who moved it here: chris@bigballofwax.co.nz.
The original submitter of attachment 451 [details] is unknown.
   Reassigning to the person who moved it here: chris@bigballofwax.co.nz.

Comment 1 Jason Sherman 2010-06-21 19:22:54 UTC Comment hidden (obsolete)
Comment 2 Jason Sherman 2010-06-21 19:24:53 UTC Comment hidden (obsolete)
Comment 3 Ian Walls 2011-12-04 16:58:22 UTC
*** Bug 1554 has been marked as a duplicate of this bug. ***
Comment 4 Sophie MEYNIEUX 2012-03-22 15:08:19 UTC Comment hidden (obsolete)
Comment 5 Kyle M Hall 2012-04-03 14:56:09 UTC Comment hidden (obsolete)
Comment 6 Ian Walls 2012-04-03 15:59:18 UTC
This patch doesn't seem to address the security concerns raised by Atz in the last imported comment... could someone speak to this before it is processed through QA?  What precautions are taken?  What vulnerabilities are possible?
Comment 7 Vitor Fernandes 2012-11-27 11:02:08 UTC
This patch seems to have a little error:

my $base = C4::Context->config('intranetdir');
...
$command .= $base . "/misc/cronjobs/runreport.pl";

So the the $command will be ".../koha/intranet/misc/cronjobs/runreport.pl" which doesn't exist.

I think this is the correct code:

$command .= $base . "../bin/cronjobs/runreport.pl";

So the the $command will be ".../koha/intranet/../bin/cronjobs/runreport.pl", pointing exactly to existent run report script.

Right?
Comment 8 MJ Ray (software.coop) 2012-12-11 15:19:06 UTC
(In reply to comment #6)
> This patch doesn't seem to address the security concerns raised by Atz in
> the last imported comment... could someone speak to this before it is
> processed through QA?  What precautions are taken?  What vulnerabilities are
> possible?

Surely that shouldn't block the improvement patch?

The last comment from Atz was "Please note that there are big security issues for using "at" (or in this case "atq") in a multiple-client hosting environment, since each Koha will be running as the same "apache" user.  There is no differentiation on what jobs are visible: everybody will see everything."

That has never been best practice and I'd be surprised and disappointed if liblime did it.

The current packages for debian use apache-mpm-itk to constrain each individual vhost to a particular system user, so the above vulnerability doesn't occur in that way. Other installers could use the same MPM, or they could use other apache modules like mod_ruid to achieve a similar isolation.

If all virtualhosts were running as one user, the vulnerability would have meant that libraries could mess with each others' jobs - deleting, amending, adding jobs for other libraries - but they've probably also got other security vulnerabilities that would arise from the lack of isolation.  I suspect most people who install it themselves and don't follow best practice are only hosting one library, in which case the vulnerability wouldn't occur because there are simply no other libraries to mess with.

That's probably a QA or RM decision that boils down to: do we design Koha to work well under best practice, or not implement a feature because someone could cause a security flaw with it?

But I'm not sure the security concern applies if installed in as suggested by any INSTALL* files, does it?  I think the only installation method that addresses multiple Koha hostings are the packages for debian and they seem safe.

The later comment about using cron is sound, but I think C4::Scheduler uses cron when appropriate now anyway.  It may be better to implement C4::Scheduler with a cron job that uses koha-foreach or multiple user crontabs to review the scheduled tasks every 10 minutes or something, but the current system-level method doesn't seem awful.
Comment 9 Katrin Fischer 2012-12-28 22:48:40 UTC
Please provide a test plan, problem description and some notes on how this patch is solving the problem. 

Also, I would like to see Vitor's question addressed (comment 7).
Comment 10 Katrin Fischer 2012-12-29 09:25:29 UTC
Hi Sophie, I missed the notes on the patch yesterday and I think I have some idea now what the patch does, but test plans are a requirement.
Comment 11 Sophie MEYNIEUX 2013-03-19 17:22:16 UTC
Test plan (as illustrated by Chris screenshots)
- Tools > Task scheduler
  or
  Reports > Use saved > Schedule (on the line of the report you want tu run)
- Select time, date, report to run (if not already chosen), output format, mail to send the result.
- Click on Save

Scheduled report is added to the table of existing scheduled reports under the form.

Without the patch, after planned time, if you refresh the page, scheduled report is not displaied anymore, but report is not run and no mail is sent.

With the patch, after planned time, recipient receive a mail containing the result of the report in plain text or attached as csv file dépending of choosen option.
Comment 12 Kyle M Hall 2013-03-20 12:53:21 UTC
I tried to rebase, but I am unsure of some of the conflicts.
Comment 13 mathieu saby 2013-03-21 20:21:29 UTC
I see a pacth signed off (in 2012??), and status does not apply. Kyle, did you sign it off after your last comment?

Mathieu Saby
Comment 14 Katrin Fischer 2013-03-22 06:04:59 UTC
Sophie, is there time to rebase this? If the scheduler is broken in master currently this could classify as a bug fix.
Comment 15 Sophie MEYNIEUX 2013-03-22 08:51:48 UTC Comment hidden (obsolete)
Comment 16 Sophie MEYNIEUX 2013-03-22 08:54:17 UTC
Patch rebased
Comment 17 mathieu saby 2013-03-22 13:05:54 UTC
I confirm scheduler is broken in master, and probably in 3.8. We never could use it in Rennes 2!

M. Saby
Comment 18 Katrin Fischer 2013-04-07 17:04:38 UTC
The QA script reports problems:

 * misc/cronjobs/runreport.pl                                               FAIL
	pod                         OK
	forbidden patterns          OK
	valid                       FAIL
		"my" variable $sql masks earlier declaration in same scope 
		"my" variable $type masks earlier declaration in same scope 
		"my" variable $report_name masks earlier declaration in same scope 

Continuing testing for now.
Comment 19 Katrin Fischer 2013-04-07 17:12:37 UTC
When trying to schedule an error message is shown:
Failed to add scheduled task

And this is in the logs:
[Sun Apr 07 19:06:15 2013] [error] [client 127.0.0.1] You do not have permission to use atq., referer: http://localhost:8080/cgi-bin/koha/tools/scheduler.pl
[Sun Apr 07 19:06:15 2013] [error] [client 127.0.0.1] You do not have permission to use atq., referer: http://localhost:8080/cgi-bin/koha/tools/scheduler.pl
Comment 20 Sophie MEYNIEUX 2013-05-07 11:57:14 UTC
Katrin, you need to give permission to your apache user to use at

On your server, you should have a file /etc/at.allow containing the name of this user (www-data most of the time) and/or a file /etc/at.deny *not* containing the name of you user

I'll take care to provide a new patch for QA issues
Comment 21 Katrin Fischer 2013-05-07 12:06:09 UTC
Hi Sophie, switch back to signed off when the follow-up patch is ready for testing. Thx!
Comment 22 Sophie MEYNIEUX 2013-05-07 14:40:29 UTC Comment hidden (obsolete)
Comment 23 Sophie MEYNIEUX 2013-05-07 15:11:52 UTC
I have squashed the previously signed off patch and my followup
This latter modifies get_saved_report function call whose API has changed
Comment 24 Katrin Fischer 2013-05-07 15:25:22 UTC
Hi Sophie,
please don't squash signed off patches together with new changes. This way we don't know what was texted by the sign offer and the sign offer also might not be happy with it :)
Comment 25 Katrin Fischer 2013-05-07 15:26:14 UTC
That said - can you maybe provide the original patches? The signed off patch could be unobsoleted, but not sure about how to divide out the newer changes.
Comment 26 Sophie MEYNIEUX 2013-05-10 13:01:04 UTC
Created attachment 18063 [details] [review]
Bug 1993: Improving task scheduler

Remove "url" as format as it is not implemented
    Add csv as output format
    Use MIME:Lite to send email
    Call runreport.pl with right parameters

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Comment 27 Sophie MEYNIEUX 2013-05-10 13:08:29 UTC
Finally, patch only needed to be rebased.
But now I don't know iln which status I should put it
Comment 28 Katrin Fischer 2013-05-10 13:12:19 UTC
Hi Sophie,
signed off is ok if it was signed off before and only needed a rebase.
Comment 29 Katrin Fischer 2013-05-17 06:32:54 UTC
Hi Sophie,

I am at a loss here... I don't know how to fix the problem with permissions on running at. I have added the users to a newly created at.allow file, but it's just not working. :(
Comment 30 Katrin Fischer 2013-05-17 06:44:43 UTC
Created attachment 18183 [details]
screenshot

Actually, the logs show no new permission errors now, but I still get the same message in the interface about the job not being added (see screenshot)
Comment 31 Marcel de Rooy 2013-05-17 07:58:12 UTC
(In reply to comment #29)
> I am at a loss here... I don't know how to fix the problem with permissions
> on running at. I have added the users to a newly created at.allow file, but
> it's just not working. :(

If you have Linux distro with SELinux enforced, that could be a reason too. In that case you need to tell SELinux too that this is okay..
Comment 32 Katrin Fischer 2013-05-17 08:30:08 UTC
I am using Ubuntu and didn't activate anything. Maybe it's realted to the form that has changed between the screenshots and the current version in master?
The error from the logs is gone, so I think I must have solved the permission problem, but I still can't schedule anything, which makes me think that there is something wrong with my input or something else inside Koha?
Comment 33 Marcel de Rooy 2013-05-17 08:34:23 UTC
Katrin:
Just another observation. I set my SELinux temporarily to Permissive. I can add a job, but there is another security issue.
At job run time, I receive the mail: This account is currently not available.
Why? Because my apache user is not allowed to login. So the job cannot be run, although it is allowed via at.allow etc.

This brings me to a more important question: Should we encourage Koha users to allow login for apache user to make use of the Task Scheduler? Note that is a general Apache hardening measure to not allow interactive login for the apache user. IMO we (as Koha developers) should not stimulate Koha users to lower security barriers to enable Koha functionality.

Another approach to the Task Scheduler could be to have a specific cronjob, look for Koha reports to run at specified times without allowing apache to add generic jobs with all security risks attached..

Moving this report to In Discussion. Will send a mail to the dev list.
Comment 34 Galen Charlton 2013-05-20 21:17:02 UTC
(In reply to comment #33)
> This brings me to a more important question: Should we encourage Koha users
> to allow login for apache user to make use of the Task Scheduler? Note that
> is a general Apache hardening measure to not allow interactive login for the
> apache user. IMO we (as Koha developers) should not stimulate Koha users to
> lower security barriers to enable Koha functionality.

I agree: we should not be asking users to grant special privileges to the user that Apache runs as,

> Another approach to the Task Scheduler could be to have a specific cronjob,
> look for Koha reports to run at specified times without allowing apache to
> add generic jobs with all security risks attached..

I think writing a cronjob (and abandoning Schedule::At) is the way to go.  In particular, I suggest the following implementation:

- create a database table to store a queue of reports to run (or maybe generalize it a bit and call it a job queue table)
- the current report scheduling functionality would just add rows to the queue with a run start time set by the user
- the cronjob would periodically check the queue and run any pending reports whose run start time has passed

This would be pretty light-weight, but would allow us to get fancier in the future.  For example, adding functionality to schedule reports to run periodically could be done by writing code to add entries into the job queue.
Comment 35 Katrin Fischer 2013-06-16 06:47:27 UTC
I am not sure how we continue best here. I think in the past it was at least possible to make the scheduler work by changing the permissions on sysadmin level. I know that this has not been recommended for security reasons.

I wonder that now, that this seems not to be working at all, we should hide (not delete) the options from the user interface until we can find someone who sponsors the rewrite.
Comment 36 mathieu saby 2013-08-19 13:06:18 UTC
Do you know if this feature really work in the past?
I find strange to pay again to fix a feature broken by somebody's code 2 or 3 years ago. Is not it really possible to fix this feature, and to make an other bug for improving safety ?

M. Saby
Comment 37 MJ Ray (software.coop) 2013-08-21 09:27:19 UTC
"I find strange to pay again to fix a feature broken by somebody's code 2 or 3 years ago." - ah well, this happens with most software, where a feature is added without a good-enough test set for it, or where some other design change is seen as desirable enough to be breaking other features, which is maybe what happened here (security is good).  Customers are paying for this sort of maintenance in other software - some developers joke "plan to throw one away - you will anyway" - and it's just more visible in open software like Koha.

So if this feature is readded (maybe with a task-runner script something like drupal's cron.php, maybe with a proper cronjob), it should be added with some test to help avoid breaking it again.
Comment 38 Nicholas van Oudtshoorn 2013-08-22 07:30:00 UTC
Hopefully this is the right bug to write about this in. Just noticed that runreport.pl (which I'm calling manually from a cron job), whilst allowing for a format of html, doesn't set the "Content-Type" on such emails to "text/html".

A simple patch replaces:

    if ($email){
        my %mail = (
            To      => $to,
            From    => $from,
            Subject => encode('utf8', $subject ),
            Message => encode('utf8', $message )
        );
        sendmail(%mail) or carp 'mail not sent:' . $Mail::Sendmail::error;
    } else {
        print $message;
    }

with
    if ($email){
        my %mail;
        if ($format eq 'html') {
                $message = "<html><head><style>tr:nth-child(n+1) { background-color: #ccc;}</style></head><body>$message</body></html>";
           %mail = (
              To      => $to,
              From    => $from,
              'Content-Type' => 'text/html',
              Subject => encode('utf8', $subject ),
              Message => encode('utf8', $message )
          );
        } else {
          %mail = (
              To      => $to,
              From    => $from,
              Subject => encode('utf8', $subject ),
              Message => encode('utf8', $message )
          );
        }
        sendmail(%mail) or carp 'mail not sent:' . $Mail::Sendmail::error;
    } else {
        print $message;
    }
Comment 39 Marcel de Rooy 2013-08-22 07:44:40 UTC
(In reply to Nicholas van Oudtshoorn from comment #38)
> Hopefully this is the right bug to write about this in. Just noticed that
> runreport.pl (which I'm calling manually from a cron job), whilst allowing
> for a format of html, doesn't set the "Content-Type" on such emails to
> "text/html".
> 

I am not sure about that, Nicholas.
This bug is about scheduling running reports, not about mailing in html format.
If you can't find a more specific bug, please open a new report for it.
And your patch is welcome too. Preferably as formatted by git. (Look for instructions on using git-bz; that will make it easier for you.)
Thanks.
Comment 40 Nicholas van Oudtshoorn 2013-08-22 08:32:34 UTC
Sorry about the noise. Have created the new bug report (10777), and will upload patch there shortly!
Comment 41 Marcel de Rooy 2013-08-22 08:38:18 UTC
(In reply to Nicholas van Oudtshoorn from comment #40)
> Sorry about the noise. Have created the new bug report (10777), and will
> upload patch there shortly!

Thx
Comment 42 Chris Cormack 2013-08-22 08:39:27 UTC
(In reply to Nicholas van Oudtshoorn from comment #40)
> Sorry about the noise. Have created the new bug report (10777), and will
> upload patch there shortly!

A patch more than makes up for it, people can make as much noise as they want, if they also send patches ;)
Comment 43 Sarwar 2013-12-05 06:43:54 UTC
Dear Sophie,

I have made changes in the koha files according to the patch you had attached in this thread.  But emails are still not generating and recipient does not receive any emails. I have also checked the at.allow and at.deny file. my at.deny file is empty that , according to my understanding , means all the users have the permissions to run task scheduler. Kindly help me with the solution ,as i am stuck on this from last few days.

When i schedule a task using task scheduler. It showed me the following command under the action tab.

export KOHA_CONF="/etc/koha/koha-conf.xml"; /usr/share/koha/intranet/cgi-bin/misc/cronjobs/runreport.pl --format csv --to ghulam.sarwar@techlogix.com 6


Looking forward to your reply.

(In reply to Sophie MEYNIEUX from comment #11)
> Test plan (as illustrated by Chris screenshots)
> - Tools > Task scheduler
>   or
>   Reports > Use saved > Schedule (on the line of the report you want tu run)
> - Select time, date, report to run (if not already chosen), output format,
> mail to send the result.
> - Click on Save
> 
> Scheduled report is added to the table of existing scheduled reports under
> the form.
> 
> Without the patch, after planned time, if you refresh the page, scheduled
> report is not displaied anymore, but report is not run and no mail is sent.
> 
> With the patch, after planned time, recipient receive a mail containing the
> result of the report in plain text or attached as csv file dépending of
> choosen option.
Comment 44 wajasu 2014-03-04 17:47:56 UTC
I got it to work by changing the command definition in tools/scheduler.pl

    my $command =
        "export KOHA_CONF=\"$CONFIG_NAME\"; export PERL5LIB=" . $base . "; " . $base
      . "/misc/cronjobs/runreport.pl ";
    $command = $command . "--format=$format " if $format;
    $command = $command . "--to='$email' " if $email;
    $command = $command . $report if $report;

AND also added www-data, which was defined in /etc/apache2/envvars (APACHE_RUN_USER) to the /etc/at.allow and no more permission problem

I had created an sql report, and did not test email, though without email the report output was seen in the mail box for the www-data user.
(su - www-data     and then 'mail'   and  p for print. )

I am just documenting my cursory findings.
Comment 45 David Cook 2014-03-13 02:57:50 UTC
I like Galen's idea of using a database table to store reports, and then using a cronjob to execute those reports on a periodic basis.

I'm not sure if we even need to allow them to specify an exact time though. One of our libraries mentioned wanting to have daily/weekly/monthly reports. It would probably be enough to set up three cronjobs (daily, weekly, monthly), and then run the reports (or jobs) that have been designated as such in the DB table.

After all, if the report is run before the library opens or after it closes, it should provide all the functionality desired. I would have my doubts about needing a report run at 1:35pm each day.

I suppose it is possible though that you might want "morning" and "afternoon" reports, so maybe it is worth using times in the DB table. Or perhaps that's where you just add another cronjob daily cronjob for a different time. 

That would give the sysadmin ultimate say while also giving librarians the overall functionality that they desire.

Robin also suggested using the 'batch' command which "executes commands when system load levels permit" (source: man batch (1) ).
Comment 46 Binu Thomas 2014-06-16 07:07:13 UTC
I was able to make it work using Wajasu's method. But the html is still not working. Requires re-write of scheduler.pl CGI.
Comment 47 Magnus Enger 2015-10-30 09:11:57 UTC
This bug might be related: 

Bug 14712 - Feature for controlling cron jobs/cmd line scripts from staff client
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=14712

There are people in Sweden working on it.
Comment 48 David Cook 2016-02-16 03:48:22 UTC
I'm also working on a related feature via my work on bug 10662.

My work is also sponsored by folks in Sweden, so hurray Sweden!

I've been thinking about opening a new bug, or looking up one of these bugs to post patches.

Basically, I've written a task scheduler, which accepts a few variables (e.g. start_time, repeat_interval, task_type, params). It then uses plugin modules to run the actual task. I've already written plugins to download records via OAI-PMH, and to upload those records to a new Koha API for OAI-PMH records specifically. It would be fairly trivial to create a "Report" plugin. 

The web user interface relies on a template include which is matched to the plugin.

The whole thing needs some work, but I'm hoping folks will test it and I'm looking at polishing it up...

At the moment, it's all at bug 10662, but I'm thinking of breaking into different bug reports...
Comment 49 Cab Vinton 2017-02-28 23:10:51 UTC
This is a minor issue, but trivial to implement once the gory work on the scheduler itself is complete -- 

When scheduling reports the list of reports in the drop-down menu should be alphabetized.

Most libraries have a large inventory of reports, so having them in alphabetical order makes a lot of sense.
Comment 50 Cab Vinton 2018-07-26 22:00:43 UTC
*** Bug 20673 has been marked as a duplicate of this bug. ***
Comment 51 David Cook 2018-08-13 01:05:20 UTC
After seeing a comment on #3935, I was thinking that maybe this is worth another revisit sometime soon.

There are lots of pre-existing job queues and task schedulers out there in the world, but - considering our embracing of the Mojolicious framework - maybe we should opt for Minion (https://mojolicious.org/perldoc/Minion). Martin Renvoize has advocated this in the past and has some experience with it, I think.

For #10662, I wrote a job queue / task scheduler using https://metacpan.org/pod/POE::Component::JobQueue and https://metacpan.org/pod/POE::Wheel::Run. I had looked at Minion a bit, but I think we hadn't accepted Mojolicious yet, and I had some requirements (like deleting already enqueued jobs before they had a chance to run) which it didn't seem to have.

I think the hardest thing for this issue is not the work itself... but making a choice in a sea of possibilities.

On that note, maybe this is an issue that would best be addressed using plugins. We could create a uniform task scheduler user interface and internal API, and then the "hard work" of interfacing with a particular scheduling system could be left up to a plugin.

That also means that we could experiment a bit before forging ahead with "The One True Way"?