Bug 30410 - Add a way for plugins to register background tasks
Summary: Add a way for plugins to register background tasks
Status: CLOSED FIXED
Alias: None
Product: Koha
Classification: Unclassified
Component: Plugin architecture (show other bugs)
Version: Main
Hardware: All All
: P5 - low enhancement
Assignee: Tomás Cohen Arazi (tcohen)
QA Contact: Kyle M Hall (khall)
URL:
Keywords:
Depends on: 27783
Blocks: 35070 32561
  Show dependency treegraph
 
Reported: 2022-03-30 15:05 UTC by Tomás Cohen Arazi (tcohen)
Modified: 2023-10-17 01:32 UTC (History)
4 users (show)

See Also:
Change sponsored?: ---
Patch complexity: ---
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:
22.05.00
Circulation function:


Attachments
Bug 30410: Unit tests (3.75 KB, patch)
2022-04-01 07:19 UTC, Tomás Cohen Arazi (tcohen)
Details | Diff | Splinter Review
Bug 30410: Add a way for plugins to expose tasks they implement (3.65 KB, patch)
2022-04-01 07:19 UTC, Tomás Cohen Arazi (tcohen)
Details | Diff | Splinter Review
Bug 30410: Unit tests (3.82 KB, patch)
2022-04-11 15:34 UTC, Martin Renvoize (ashimema)
Details | Diff | Splinter Review
Bug 30410: Add a way for plugins to expose tasks they implement (3.71 KB, patch)
2022-04-11 15:34 UTC, Martin Renvoize (ashimema)
Details | Diff | Splinter Review
Bug 30410: Unit tests (3.81 KB, patch)
2022-04-11 16:57 UTC, Kyle M Hall (khall)
Details | Diff | Splinter Review
Bug 30410: Add a way for plugins to expose tasks they implement (3.71 KB, patch)
2022-04-11 16:57 UTC, Kyle M Hall (khall)
Details | Diff | Splinter Review
Bug 30410: (QA follow-up) Pluralise new methods, fix minor issues with language (3.47 KB, patch)
2022-04-11 16:57 UTC, Kyle M Hall (khall)
Details | Diff | Splinter Review
Bug 30410: (QA follow-up) Only fetch plugins mapping once (806 bytes, patch)
2022-04-11 17:03 UTC, Kyle M Hall (khall)
Details | Diff | Splinter Review
Bug 30410: (follow-up) Instantiate object in unit test (1.14 KB, patch)
2022-04-19 07:33 UTC, Martin Renvoize (ashimema)
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Tomás Cohen Arazi (tcohen) 2022-03-30 15:05:30 UTC
The idea is to have a way for plugins to define 'code' => 'class' mappings so the worker(s) can process tasks through plugin-provided classes.
Comment 1 Tomás Cohen Arazi (tcohen) 2022-04-01 07:19:53 UTC
Created attachment 132814 [details] [review]
Bug 30410: Unit tests
Comment 2 Tomás Cohen Arazi (tcohen) 2022-04-01 07:19:57 UTC
Created attachment 132815 [details] [review]
Bug 30410: Add a way for plugins to expose tasks they implement

This patch implements a mechanism for plugins to tell Koha they
implement background jobs, by returning a mapping

```
    task => class
```

* _task_ is a string that will be used when queueing new tasks, to let
  the workers know which class to instantiate to launch the job (a.k.a.
  process).

* _class_ is a string for a class name.

For this to  work, plugins need to include the 'namespace' attribute in
their metadata. If they don't, they will be skipped when listing the
valid _background jobs_.

After this, high-level methods for letting plugins (easily) enqueue
their tasks will be provided.

To test:
1. Apply this patch
2. Run:
   $ kshell
  k$ prove t/db_dependent/Koha/Plugins/BackgroundJob.t
=> SUCCESS: Tests pass!
3. Sign off :-D

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Comment 3 David Cook 2022-04-03 23:54:10 UTC
(In reply to Tomás Cohen Arazi from comment #2)
> For this to  work, plugins need to include the 'namespace' attribute in
> their metadata. If they don't, they will be skipped when listing the
> valid _background jobs_.

What's an example 'namespace' attribute? Is that used to prevent collisions with task names?
Comment 4 Tomás Cohen Arazi (tcohen) 2022-04-04 06:06:26 UTC
(In reply to David Cook from comment #3)
> (In reply to Tomás Cohen Arazi from comment #2)
> > For this to  work, plugins need to include the 'namespace' attribute in
> > their metadata. If they don't, they will be skipped when listing the
> > valid _background jobs_.
> 
> What's an example 'namespace' attribute? Is that used to prevent collisions
> with task names?

There's a clear example in the patches.
Thanks
Comment 5 David Cook 2022-04-04 06:36:22 UTC
(In reply to Tomás Cohen Arazi from comment #4)
> There's a clear example in the patches.
> Thanks

My bad. I had already read the main patch but not the unit tests patch. I should've thought to look there, although I didn't think the example was that clear. I had to re-read it a few times.

I suppose the plugin's "name" isn't usable since it's human-friendly and not machine friendly. 

A default method inherited from Koha::Plugins::Base that does something like this might be useful:

sub namespace {
  my $namespace = lc(__PACKAGE__);
  $namespace =~ s/::/_/g
  return $namespace
}

I suppose free text and auto-generated both have their pros and cons...
Comment 6 Tomás Cohen Arazi (tcohen) 2022-04-04 23:08:43 UTC
(In reply to David Cook from comment #5)
> (In reply to Tomás Cohen Arazi from comment #4)
> > There's a clear example in the patches.
> > Thanks
> 
> I suppose the plugin's "name" isn't usable since it's human-friendly and not
> machine friendly. 
> 
> I suppose free text and auto-generated both have their pros and cons...

The main problem is how big the column needs to be.
I suggest you go try and sign the patches. Thanks!
Comment 7 Martin Renvoize (ashimema) 2022-04-11 15:34:26 UTC
Created attachment 133180 [details] [review]
Bug 30410: Unit tests

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 8 Martin Renvoize (ashimema) 2022-04-11 15:34:30 UTC
Created attachment 133181 [details] [review]
Bug 30410: Add a way for plugins to expose tasks they implement

This patch implements a mechanism for plugins to tell Koha they
implement background jobs, by returning a mapping

```
    task => class
```

* _task_ is a string that will be used when queueing new tasks, to let
  the workers know which class to instantiate to launch the job (a.k.a.
  process).

* _class_ is a string for a class name.

For this to  work, plugins need to include the 'namespace' attribute in
their metadata. If they don't, they will be skipped when listing the
valid _background jobs_.

After this, high-level methods for letting plugins (easily) enqueue
their tasks will be provided.

To test:
1. Apply this patch
2. Run:
   $ kshell
  k$ prove t/db_dependent/Koha/Plugins/BackgroundJob.t
=> SUCCESS: Tests pass!
3. Sign off :-D

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 9 Martin Renvoize (ashimema) 2022-04-11 15:35:11 UTC
Nice clear code and gives us a great way to develop background tasks.

All works as expected, Signing off
Comment 10 Kyle M Hall (khall) 2022-04-11 16:57:19 UTC
Created attachment 133192 [details] [review]
Bug 30410: Unit tests

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Comment 11 Kyle M Hall (khall) 2022-04-11 16:57:53 UTC
Created attachment 133193 [details] [review]
Bug 30410: Add a way for plugins to expose tasks they implement

This patch implements a mechanism for plugins to tell Koha they
implement background jobs, by returning a mapping

```
    task => class
```

* _task_ is a string that will be used when queueing new tasks, to let
  the workers know which class to instantiate to launch the job (a.k.a.
  process).

* _class_ is a string for a class name.

For this to  work, plugins need to include the 'namespace' attribute in
their metadata. If they don't, they will be skipped when listing the
valid _background jobs_.

After this, high-level methods for letting plugins (easily) enqueue
their tasks will be provided.

To test:
1. Apply this patch
2. Run:
   $ kshell
  k$ prove t/db_dependent/Koha/Plugins/BackgroundJob.t
=> SUCCESS: Tests pass!
3. Sign off :-D

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Comment 12 Kyle M Hall (khall) 2022-04-11 16:57:57 UTC
Created attachment 133194 [details] [review]
Bug 30410: (QA follow-up) Pluralise new methods, fix minor issues with language

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Comment 13 Kyle M Hall (khall) 2022-04-11 17:03:06 UTC
Created attachment 133195 [details] [review]
Bug 30410: (QA follow-up) Only fetch plugins mapping once
Comment 14 Fridolin Somers 2022-04-13 12:51:32 UTC
Please add to https://wiki.koha-community.org/wiki/Koha_Plugin_Hooks
Comment 15 Fridolin Somers 2022-04-13 13:58:00 UTC
Pushed to master for 22.05, thanks to everybody involved ﷐[U+1F984]﷑
Comment 16 Martin Renvoize (ashimema) 2022-04-19 07:33:24 UTC
Created attachment 133386 [details] [review]
Bug 30410: (follow-up) Instantiate object in unit test

This patch updates the unit test for the BackgroundJob->_derived_class()
private method to instantiate a proper object to work on instead of
calling the type_to_class_mapping method directly as non-object method.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 17 Fridolin Somers 2022-04-19 22:06:47 UTC
Pushed to master :
  Bug 30410: (follow-up) Instantiate object in unit test
Comment 18 David Cook 2023-01-09 05:12:53 UTC
Do we have any examples of this one?
Comment 19 Fridolin Somers 2023-01-17 03:15:57 UTC
Wiki page updated
Comment 20 David Cook 2023-01-17 04:43:40 UTC
(In reply to Fridolin Somers from comment #19)
> Wiki page updated

Thanks, Frido, although I don't see any example plugins?
Comment 21 Fridolin Somers 2023-01-18 06:09:07 UTC
I had a look but I really dont understand how to implement the hook.
A pool request on kitchen sink is much welcomed.
Comment 22 David Cook 2023-01-18 06:15:38 UTC
(In reply to Fridolin Somers from comment #21)
> I had a look but I really dont understand how to implement the hook.
> A pool request on kitchen sink is much welcomed.

Ok. I'm on leave the next 4 days, but I'll hopefully look at creating a plugin next week (if I can figure out how to!)
Comment 23 David Cook 2023-10-13 05:42:14 UTC
(In reply to Tomás Cohen Arazi from comment #4)
> There's a clear example in the patches.

(In reply to Martin Renvoize from comment #9)
> Nice clear code and gives us a great way to develop background tasks.

After nearly a year, I still can't understand this code and have no idea how to develop a plugin to use this hook.

I'm keen to try it out, so I'll keep trying to figure it out, but it would be great to have some assistance either here or on the Koha Plugin Hooks wiki page...