Bug 30410 - Add a way for plugins to register background tasks
Summary: Add a way for plugins to register background tasks
Status: RESOLVED FIXED
Alias: None
Product: Koha
Classification: Unclassified
Component: Plugin architecture (show other bugs)
Version: master
Hardware: All All
: P5 - low enhancement (vote)
Assignee: Tomás Cohen Arazi
QA Contact: Kyle M Hall
URL:
Keywords:
Depends on: 27783
Blocks:
  Show dependency treegraph
 
Reported: 2022-03-30 15:05 UTC by Tomás Cohen Arazi
Modified: 2022-04-19 22:06 UTC (History)
4 users (show)

See Also:
Change sponsored?: ---
Patch complexity: ---
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:
22.05.00


Attachments
Bug 30410: Unit tests (3.75 KB, patch)
2022-04-01 07:19 UTC, Tomás Cohen Arazi
Details | Diff | Splinter Review
Bug 30410: Add a way for plugins to expose tasks they implement (3.65 KB, patch)
2022-04-01 07:19 UTC, Tomás Cohen Arazi
Details | Diff | Splinter Review
Bug 30410: Unit tests (3.82 KB, patch)
2022-04-11 15:34 UTC, Martin Renvoize
Details | Diff | Splinter Review
Bug 30410: Add a way for plugins to expose tasks they implement (3.71 KB, patch)
2022-04-11 15:34 UTC, Martin Renvoize
Details | Diff | Splinter Review
Bug 30410: Unit tests (3.81 KB, patch)
2022-04-11 16:57 UTC, Kyle M Hall
Details | Diff | Splinter Review
Bug 30410: Add a way for plugins to expose tasks they implement (3.71 KB, patch)
2022-04-11 16:57 UTC, Kyle M Hall
Details | Diff | Splinter Review
Bug 30410: (QA follow-up) Pluralise new methods, fix minor issues with language (3.47 KB, patch)
2022-04-11 16:57 UTC, Kyle M Hall
Details | Diff | Splinter Review
Bug 30410: (QA follow-up) Only fetch plugins mapping once (806 bytes, patch)
2022-04-11 17:03 UTC, Kyle M Hall
Details | Diff | Splinter Review
Bug 30410: (follow-up) Instantiate object in unit test (1.14 KB, patch)
2022-04-19 07:33 UTC, Martin Renvoize
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Tomás Cohen Arazi 2022-03-30 15:05:30 UTC
The idea is to have a way for plugins to define 'code' => 'class' mappings so the worker(s) can process tasks through plugin-provided classes.
Comment 1 Tomás Cohen Arazi 2022-04-01 07:19:53 UTC
Created attachment 132814 [details] [review]
Bug 30410: Unit tests
Comment 2 Tomás Cohen Arazi 2022-04-01 07:19:57 UTC
Created attachment 132815 [details] [review]
Bug 30410: Add a way for plugins to expose tasks they implement

This patch implements a mechanism for plugins to tell Koha they
implement background jobs, by returning a mapping

```
    task => class
```

* _task_ is a string that will be used when queueing new tasks, to let
  the workers know which class to instantiate to launch the job (a.k.a.
  process).

* _class_ is a string for a class name.

For this to  work, plugins need to include the 'namespace' attribute in
their metadata. If they don't, they will be skipped when listing the
valid _background jobs_.

After this, high-level methods for letting plugins (easily) enqueue
their tasks will be provided.

To test:
1. Apply this patch
2. Run:
   $ kshell
  k$ prove t/db_dependent/Koha/Plugins/BackgroundJob.t
=> SUCCESS: Tests pass!
3. Sign off :-D

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Comment 3 David Cook 2022-04-03 23:54:10 UTC
(In reply to Tomás Cohen Arazi from comment #2)
> For this to  work, plugins need to include the 'namespace' attribute in
> their metadata. If they don't, they will be skipped when listing the
> valid _background jobs_.

What's an example 'namespace' attribute? Is that used to prevent collisions with task names?
Comment 4 Tomás Cohen Arazi 2022-04-04 06:06:26 UTC
(In reply to David Cook from comment #3)
> (In reply to Tomás Cohen Arazi from comment #2)
> > For this to  work, plugins need to include the 'namespace' attribute in
> > their metadata. If they don't, they will be skipped when listing the
> > valid _background jobs_.
> 
> What's an example 'namespace' attribute? Is that used to prevent collisions
> with task names?

There's a clear example in the patches.
Thanks
Comment 5 David Cook 2022-04-04 06:36:22 UTC
(In reply to Tomás Cohen Arazi from comment #4)
> There's a clear example in the patches.
> Thanks

My bad. I had already read the main patch but not the unit tests patch. I should've thought to look there, although I didn't think the example was that clear. I had to re-read it a few times.

I suppose the plugin's "name" isn't usable since it's human-friendly and not machine friendly. 

A default method inherited from Koha::Plugins::Base that does something like this might be useful:

sub namespace {
  my $namespace = lc(__PACKAGE__);
  $namespace =~ s/::/_/g
  return $namespace
}

I suppose free text and auto-generated both have their pros and cons...
Comment 6 Tomás Cohen Arazi 2022-04-04 23:08:43 UTC
(In reply to David Cook from comment #5)
> (In reply to Tomás Cohen Arazi from comment #4)
> > There's a clear example in the patches.
> > Thanks
> 
> I suppose the plugin's "name" isn't usable since it's human-friendly and not
> machine friendly. 
> 
> I suppose free text and auto-generated both have their pros and cons...

The main problem is how big the column needs to be.
I suggest you go try and sign the patches. Thanks!
Comment 7 Martin Renvoize 2022-04-11 15:34:26 UTC
Created attachment 133180 [details] [review]
Bug 30410: Unit tests

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 8 Martin Renvoize 2022-04-11 15:34:30 UTC
Created attachment 133181 [details] [review]
Bug 30410: Add a way for plugins to expose tasks they implement

This patch implements a mechanism for plugins to tell Koha they
implement background jobs, by returning a mapping

```
    task => class
```

* _task_ is a string that will be used when queueing new tasks, to let
  the workers know which class to instantiate to launch the job (a.k.a.
  process).

* _class_ is a string for a class name.

For this to  work, plugins need to include the 'namespace' attribute in
their metadata. If they don't, they will be skipped when listing the
valid _background jobs_.

After this, high-level methods for letting plugins (easily) enqueue
their tasks will be provided.

To test:
1. Apply this patch
2. Run:
   $ kshell
  k$ prove t/db_dependent/Koha/Plugins/BackgroundJob.t
=> SUCCESS: Tests pass!
3. Sign off :-D

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 9 Martin Renvoize 2022-04-11 15:35:11 UTC
Nice clear code and gives us a great way to develop background tasks.

All works as expected, Signing off
Comment 10 Kyle M Hall 2022-04-11 16:57:19 UTC
Created attachment 133192 [details] [review]
Bug 30410: Unit tests

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Comment 11 Kyle M Hall 2022-04-11 16:57:53 UTC
Created attachment 133193 [details] [review]
Bug 30410: Add a way for plugins to expose tasks they implement

This patch implements a mechanism for plugins to tell Koha they
implement background jobs, by returning a mapping

```
    task => class
```

* _task_ is a string that will be used when queueing new tasks, to let
  the workers know which class to instantiate to launch the job (a.k.a.
  process).

* _class_ is a string for a class name.

For this to  work, plugins need to include the 'namespace' attribute in
their metadata. If they don't, they will be skipped when listing the
valid _background jobs_.

After this, high-level methods for letting plugins (easily) enqueue
their tasks will be provided.

To test:
1. Apply this patch
2. Run:
   $ kshell
  k$ prove t/db_dependent/Koha/Plugins/BackgroundJob.t
=> SUCCESS: Tests pass!
3. Sign off :-D

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Comment 12 Kyle M Hall 2022-04-11 16:57:57 UTC
Created attachment 133194 [details] [review]
Bug 30410: (QA follow-up) Pluralise new methods, fix minor issues with language

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Comment 13 Kyle M Hall 2022-04-11 17:03:06 UTC
Created attachment 133195 [details] [review]
Bug 30410: (QA follow-up) Only fetch plugins mapping once
Comment 14 Fridolin Somers 2022-04-13 12:51:32 UTC
Please add to https://wiki.koha-community.org/wiki/Koha_Plugin_Hooks
Comment 15 Fridolin Somers 2022-04-13 13:58:00 UTC
Pushed to master for 22.05, thanks to everybody involved 🦄
Comment 16 Martin Renvoize 2022-04-19 07:33:24 UTC
Created attachment 133386 [details] [review]
Bug 30410: (follow-up) Instantiate object in unit test

This patch updates the unit test for the BackgroundJob->_derived_class()
private method to instantiate a proper object to work on instead of
calling the type_to_class_mapping method directly as non-object method.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 17 Fridolin Somers 2022-04-19 22:06:47 UTC
Pushed to master :
  Bug 30410: (follow-up) Instantiate object in unit test