Bug 25460 - Allow using item information in OAI set mappings and automatically update sets when items are added, edited or deleted
Summary: Allow using item information in OAI set mappings and automatically update set...
Status: Pushed to master
Alias: None
Product: Koha
Classification: Unclassified
Component: Web services (show other bugs)
Version: master
Hardware: All All
: P5 - low enhancement (vote)
Assignee: Alex Buckley
QA Contact: Testopia
URL:
Keywords:
Depends on: 26589
Blocks:
  Show dependency treegraph
 
Reported: 2020-05-12 02:41 UTC by Alex Buckley
Modified: 2020-10-15 12:01 UTC (History)
1 user (show)

See Also:
Change sponsored?: Sponsored
Patch complexity: Small patch
Who signed the patch off:
Text to go in the release notes:
This allows library staff to use information from items when creating mappings for OAI sets. When the new system preference OAI-PMH:AutoUpdateSetsEmbedItemData is set, editing, deleting or adding items will update records listed in the OAI sets according to the mapppings.
Version(s) released in:
20.11.00


Attachments
Bug 25460: Update OAI sets when adding/editing/deleting item records (14.27 KB, patch)
2020-05-12 03:47 UTC, Alex Buckley
Details | Diff | Splinter Review
Bug 25460: Update OAI sets when adding/editing/deleting item records (14.47 KB, patch)
2020-05-17 22:06 UTC, Alex Buckley
Details | Diff | Splinter Review
Bug 25460: Update OAI sets when adding/editing/deleting item records (14.47 KB, patch)
2020-05-18 01:19 UTC, Alex Buckley
Details | Diff | Splinter Review
Bug 25460: Update OAI sets when adding/editing/deleting item records (13.76 KB, patch)
2020-05-18 01:57 UTC, Alex Buckley
Details | Diff | Splinter Review
Bug 25460: Update OAI sets when adding/editing/deleting item records (13.92 KB, patch)
2020-05-18 02:12 UTC, Alex Buckley
Details | Diff | Splinter Review
Bug 25460: Update OAI sets when adding/editing/deleting item records (13.93 KB, patch)
2020-09-02 21:28 UTC, Alex Buckley
Details | Diff | Splinter Review
Bug 25460: Update OAI sets when adding/editing/deleting item records (13.97 KB, patch)
2020-09-20 19:11 UTC, David Nind
Details | Diff | Splinter Review
Bug 25460 : Add unit test for change to C4::OAI::Sets->UpdateOAISetsBiblio() (2.95 KB, patch)
2020-09-24 11:37 UTC, Alex Buckley
Details | Diff | Splinter Review
Bug 25460 : Add unit test for change to C4::OAI::Sets->UpdateOAISetsBiblio() (2.78 KB, patch)
2020-10-01 23:40 UTC, Alex Buckley
Details | Diff | Splinter Review
Bug 25460: Update OAI sets when adding/editing/deleting item records (14.01 KB, patch)
2020-10-02 10:03 UTC, David Nind
Details | Diff | Splinter Review
Bug 25460 : Add unit test for change to C4::OAI::Sets->UpdateOAISetsBiblio() (2.82 KB, patch)
2020-10-02 10:03 UTC, David Nind
Details | Diff | Splinter Review
Bug 25460: Update OAI sets when adding/editing/deleting item records (14.11 KB, patch)
2020-10-14 22:02 UTC, Katrin Fischer
Details | Diff | Splinter Review
Bug 25460: Add unit test for change to C4::OAI::Sets->UpdateOAISetsBiblio() (2.88 KB, patch)
2020-10-14 22:02 UTC, Katrin Fischer
Details | Diff | Splinter Review
Bug 25460: (QA follow-up) Readd ending punctuation to sys pref description (1.21 KB, patch)
2020-10-14 22:02 UTC, Katrin Fischer
Details | Diff | Splinter Review
Bug 25460: (QA follow-up) Remove stray quote from system preference descrpition (1.21 KB, patch)
2020-10-14 22:09 UTC, Katrin Fischer
Details | Diff | Splinter Review
Bug 25460: Add unit test for change to C4::OAI::Sets->UpdateOAISetsBiblio() (3.11 KB, patch)
2020-10-15 10:05 UTC, Jonathan Druart
Details | Diff | Splinter Review
Bug 25460: (QA follow-up) Remove stray quote from system preference description (1.20 KB, patch)
2020-10-15 10:08 UTC, Jonathan Druart
Details | Diff | Splinter Review
Bug 25460: Remove swp file (36.84 KB, patch)
2020-10-15 12:01 UTC, Jonathan Druart
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Alex Buckley 2020-05-12 02:41:23 UTC
Currently the if you have a OAI set with mappings containing 952 subfields then when library staff add/edit/delete items the OAI set is not updated. 

OAI sets are only updated if:

* OAI-PMH:AutoUpdateSets syspref is enabled & biblio record is added/edited/deleted

This is because C4::OAI::Sets->UpdateOAISetsBiblio() is called from  C4::Biblio->AddBiblio() or C4::Biblio->ModBiblio() neither of which are executed on adding/editing/deleting items. 

This patch will call C4::OAI::Sets->UpdateOAISetsBiblio() when adding/editing/deleting items as well as biblios if OAI-PMH:AutoUpdateSets syspref is enabled.

It will also allow library staff to define if they want item information embedded in these automatic updates. i.e. they have mappings on 952 field and want those subfields to be checked when adding/editing/deleting items.
Comment 1 Alex Buckley 2020-05-12 03:47:01 UTC
Created attachment 104721 [details] [review]
Bug 25460: Update OAI sets when adding/editing/deleting item records

Test plan:
1. Enable OAI-PMH and OAI-PMH:AutoUpdateSets sysprefs
2. Create a biblio record with 1 item with CCODE of NFIC
3. In Administration > OAI Sets configuration create a set and define
the mapping:
Field: 952, Subfield: 8, Operator: is equal to,  Value: FIC

4. Run the build_oai_sets.pl script:
 cd misc/migration_tools
 sudo koha-shell <instancename>
 ./build_oai_sets.pl -i

5. Check what items are in the OAI set:
sudo koha-mysql <instancename>
select * from oai_sets_biblios;

6. Observe your bib from step 2 is not in the set
7. [EDIT] Edit the item to have CCODE of FIC repeat step 5 and observe biblio is still not in OAI set
8. [ADD] Add a new item with CCODE of FIC and repeat step 5 and observe biblio
still not in OAI set
9. [DELETE] Delete one of your 2 items and repeat step 5 and observe biblio still
not in OAI set

10. Apply patch, update database and restart plack (instruction below on
the latter 2 actions):

cd installer/data/mysql
sudo koha-shell <instancename>
./updatedatabase.pl
sudo koha-plack --restart <instancename>

11. Enable new syspref OAI-PMH:AutoUpdateSetsEmbedItemData
12. [ADD] Repeat step 8 and observe biblio is now in the OAI set
13. [EDIT] Edit both items to have a CCODE of NFIC and repeat step 5 and
observe bib now not in set
14. [EDIT] Edit one of the items to have CCODE of FIC and repeat step 5
and observe bib now in set
15. [DELETE] Delete all items off bib and repeat step 5 and observe bib
no longer in set

Sponsored-By: Catalyst IT
Comment 2 Alex Buckley 2020-05-17 22:06:54 UTC
Created attachment 104983 [details] [review]
Bug 25460: Update OAI sets when adding/editing/deleting item records

Test plan:
1. Enable OAI-PMH and OAI-PMH:AutoUpdateSets sysprefs
2. Create a biblio record with 1 item with CCODE of NFIC
3. In Administration > OAI Sets configuration create a set and define
the mapping:
Field: 952, Subfield: 8, Operator: is equal to,  Value: FIC

4. Run the build_oai_sets.pl script:
 cd misc/migration_tools
 sudo koha-shell <instancename>
 ./build_oai_sets.pl -i

5. Check what items are in the OAI set:
sudo koha-mysql <instancename>
select * from oai_sets_biblios;

6. Observe your bib from step 2 is not in the set
7. [EDIT] Edit the item to have CCODE of FIC repeat step 5 and observe biblio is still not in OAI set
8. [ADD] Add a new item with CCODE of FIC and repeat step 5 and observe biblio
still not in OAI set
9. [DELETE] Delete one of your 2 items and repeat step 5 and observe biblio still
not in OAI set

10. Apply patch, update database and restart plack (instruction below on
the latter 2 actions):

cd installer/data/mysql
sudo koha-shell <instancename>
./updatedatabase.pl
sudo koha-plack --restart <instancename>

11. Enable new syspref OAI-PMH:AutoUpdateSetsEmbedItemData
12. [ADD] Repeat step 8 and observe biblio is now in the OAI set
13. [EDIT] Edit both items to have a CCODE of NFIC and repeat step 5 and
observe bib now not in set
14. [EDIT] Edit one of the items to have CCODE of FIC and repeat step 5
and observe bib now in set
15. [DELETE] Delete all items off bib and repeat step 5 and observe bib
no longer in set

Sponsored-By: Catalyst IT
Comment 3 Alex Buckley 2020-05-18 01:19:43 UTC
Created attachment 104985 [details] [review]
Bug 25460: Update OAI sets when adding/editing/deleting item records

Test plan:
1. Enable OAI-PMH and OAI-PMH:AutoUpdateSets sysprefs
2. Create a biblio record with 1 item with CCODE of NFIC
3. In Administration > OAI Sets configuration create a set and define
the mapping:
Field: 952, Subfield: 8, Operator: is equal to,  Value: FIC

4. Run the build_oai_sets.pl script:
 cd misc/migration_tools
 sudo koha-shell <instancename>
 ./build_oai_sets.pl -i

5. Check what items are in the OAI set:
sudo koha-mysql <instancename>
select * from oai_sets_biblios;

6. Observe your bib from step 2 is not in the set
7. [EDIT] Edit the item to have CCODE of FIC repeat step 5 and observe biblio is still not in OAI set
8. [ADD] Add a new item with CCODE of FIC and repeat step 5 and observe biblio
still not in OAI set
9. [DELETE] Delete one of your 2 items and repeat step 5 and observe biblio still
not in OAI set

10. Apply patch, update database and restart plack (instruction below on
the latter 2 actions):

cd installer/data/mysql
sudo koha-shell <instancename>
./updatedatabase.pl
sudo koha-plack --restart <instancename>

11. Enable new syspref OAI-PMH:AutoUpdateSetsEmbedItemData
12. [ADD] Repeat step 8 and observe biblio is now in the OAI set
13. [EDIT] Edit both items to have a CCODE of NFIC and repeat step 5 and
observe bib now not in set
14. [EDIT] Edit one of the items to have CCODE of FIC and repeat step 5
and observe bib now in set
15. [DELETE] Delete all items off bib and repeat step 5 and observe bib
no longer in set

Sponsored-By: Catalyst IT
Comment 4 Alex Buckley 2020-05-18 01:57:05 UTC
Created attachment 104986 [details] [review]
Bug 25460: Update OAI sets when adding/editing/deleting item records

Test plan:
1. Enable OAI-PMH and OAI-PMH:AutoUpdateSets sysprefs
2. Create a biblio record with 1 item with CCODE of NFIC
3. In Administration > OAI Sets configuration create a set and define
the mapping:
Field: 952, Subfield: 8, Operator: is equal to,  Value: FIC

4. Run the build_oai_sets.pl script:
 cd misc/migration_tools
 sudo koha-shell <instancename>
 ./build_oai_sets.pl -i

5. Check what items are in the OAI set:
sudo koha-mysql <instancename>
select * from oai_sets_biblios;

6. Observe your bib from step 2 is not in the set
7. [EDIT] Edit the item to have CCODE of FIC repeat step 5 and observe biblio is still not in OAI set
8. [ADD] Add a new item with CCODE of FIC and repeat step 5 and observe biblio
still not in OAI set
9. [DELETE] Delete one of your 2 items and repeat step 5 and observe biblio still
not in OAI set

10. Apply patch, update database and restart plack (instruction below on
the latter 2 actions):

cd installer/data/mysql
sudo koha-shell <instancename>
./updatedatabase.pl
sudo koha-plack --restart <instancename>

11. Enable new syspref OAI-PMH:AutoUpdateSetsEmbedItemData
12. [ADD] Repeat step 8 and observe biblio is now in the OAI set
13. [EDIT] Edit both items to have a CCODE of NFIC and repeat step 5 and
observe bib now not in set
14. [EDIT] Edit one of the items to have CCODE of FIC and repeat step 5
and observe bib now in set
15. [DELETE] Delete all items off bib and repeat step 5 and observe bib
no longer in set

Sponsored-By: Catalyst IT
Comment 5 Alex Buckley 2020-05-18 02:12:52 UTC
Created attachment 104987 [details] [review]
Bug 25460: Update OAI sets when adding/editing/deleting item records

Also adds syspref to optionally choose to embed item information in record when adding/editing bibliographic and item records.

Test plan:
1. Enable OAI-PMH and OAI-PMH:AutoUpdateSets sysprefs
2. Create a biblio record with 1 item with CCODE of NFIC
3. In Administration > OAI Sets configuration create a set and define
the mapping:
Field: 952, Subfield: 8, Operator: is equal to,  Value: FIC

4. Run the build_oai_sets.pl script:
 cd misc/migration_tools
 sudo koha-shell <instancename>
 ./build_oai_sets.pl -i

5. Check what items are in the OAI set:
sudo koha-mysql <instancename>
select * from oai_sets_biblios;

6. Observe your bib from step 2 is not in the set
7. [ITEM EDIT] Edit the item to have CCODE of FIC repeat step 5 and observe biblio is still not in OAI set
8. [ITEM ADD] Add a new item with CCODE of FIC and repeat step 5 and observe biblio
still not in OAI set
9. [ITEM DELETE] Delete one of your 2 items and repeat step 5 and observe biblio still
not in OAI set

10. Apply patch, update database and restart plack (instruction below on
the latter 2 actions):

cd installer/data/mysql
sudo koha-shell <instancename>
./updatedatabase.pl
sudo koha-plack --restart <instancename>

11. Enable new syspref OAI-PMH:AutoUpdateSetsEmbedItemData
12. [ITEM ADD] Repeat step 8 and observe biblio is now in the OAI set
13. [ITEM EDIT] Edit both items to have a CCODE of NFIC and repeat step 5 and
observe bib now not in set
14. [ITEM EDIT] Edit one of the items to have CCODE of FIC and repeat step 5
and observe bib now in set
15. [ITEM DELETE] Delete all items off bib and repeat step 5 and observe bib
no longer in set

Sponsored-By: Catalyst IT
Comment 6 Alex Buckley 2020-09-02 21:28:40 UTC
Created attachment 109560 [details] [review]
Bug 25460: Update OAI sets when adding/editing/deleting item records

Also adds syspref to optionally choose to embed item information in record when adding/editing bibliographic and item records.

Test plan:
1. Enable OAI-PMH and OAI-PMH:AutoUpdateSets sysprefs
2. Create a biblio record with 1 item with CCODE of NFIC
3. In Administration > OAI Sets configuration create a set and define
the mapping:
Field: 952, Subfield: 8, Operator: is equal to,  Value: FIC

4. Run the build_oai_sets.pl script:
 cd misc/migration_tools
 sudo koha-shell <instancename>
 ./build_oai_sets.pl -i

5. Check what items are in the OAI set:
sudo koha-mysql <instancename>
select * from oai_sets_biblios;

6. Observe your bib from step 2 is not in the set
7. [ITEM EDIT] Edit the item to have CCODE of FIC repeat step 5 and observe biblio is still not in OAI set
8. [ITEM ADD] Add a new item with CCODE of FIC and repeat step 5 and observe biblio
still not in OAI set
9. [ITEM DELETE] Delete one of your 2 items and repeat step 5 and observe biblio still
not in OAI set

10. Apply patch, update database and restart plack (instruction below on
the latter 2 actions):

cd installer/data/mysql
sudo koha-shell <instancename>
./updatedatabase.pl
sudo koha-plack --restart <instancename>

11. Enable new syspref OAI-PMH:AutoUpdateSetsEmbedItemData
12. [ITEM ADD] Repeat step 8 and observe biblio is now in the OAI set
13. [ITEM EDIT] Edit both items to have a CCODE of NFIC and repeat step 5 and
observe bib now not in set
14. [ITEM EDIT] Edit one of the items to have CCODE of FIC and repeat step 5
and observe bib now in set
15. [ITEM DELETE] Delete all items off bib and repeat step 5 and observe bib
no longer in set

Sponsored-By: Catalyst IT
Comment 7 Alex Buckley 2020-09-02 21:28:54 UTC
Rebased against master
Comment 8 David Nind 2020-09-20 19:11:30 UTC
Created attachment 110435 [details] [review]
Bug 25460: Update OAI sets when adding/editing/deleting item records

Also adds syspref to optionally choose to embed item information in record when adding/editing bibliographic and item records.

Test plan:
1. Enable OAI-PMH and OAI-PMH:AutoUpdateSets sysprefs
2. Create a biblio record with 1 item with CCODE of NFIC
3. In Administration > OAI Sets configuration create a set and define
the mapping:
Field: 952, Subfield: 8, Operator: is equal to,  Value: FIC

4. Run the build_oai_sets.pl script:
 cd misc/migration_tools
 sudo koha-shell <instancename>
 ./build_oai_sets.pl -i

5. Check what items are in the OAI set:
sudo koha-mysql <instancename>
select * from oai_sets_biblios;

6. Observe your bib from step 2 is not in the set
7. [ITEM EDIT] Edit the item to have CCODE of FIC repeat step 5 and observe biblio is still not in OAI set
8. [ITEM ADD] Add a new item with CCODE of FIC and repeat step 5 and observe biblio
still not in OAI set
9. [ITEM DELETE] Delete one of your 2 items and repeat step 5 and observe biblio still
not in OAI set

10. Apply patch, update database and restart plack (instruction below on
the latter 2 actions):

cd installer/data/mysql
sudo koha-shell <instancename>
./updatedatabase.pl
sudo koha-plack --restart <instancename>

11. Enable new syspref OAI-PMH:AutoUpdateSetsEmbedItemData
12. [ITEM ADD] Repeat step 8 and observe biblio is now in the OAI set
13. [ITEM EDIT] Edit both items to have a CCODE of NFIC and repeat step 5 and
observe bib now not in set
14. [ITEM EDIT] Edit one of the items to have CCODE of FIC and repeat step 5
and observe bib now in set
15. [ITEM DELETE] Delete all items off bib and repeat step 5 and observe bib
no longer in set

Sponsored-By: Catalyst IT

Signed-off-by: David Nind <david@davidnind.com>
Comment 9 Katrin Fischer 2020-09-20 19:59:38 UTC
Hi Alex, can we please have a unit test highlighting the change?
Comment 10 Alex Buckley 2020-09-24 11:37:11 UTC
Created attachment 110645 [details] [review]
Bug 25460 : Add unit test for change to C4::OAI::Sets->UpdateOAISetsBiblio()

Test plan:

1. Run unit test:
sudo koha-shell <instancename>
prove t/db_dependent/OAI/Sets.t

2. All tests should pass

Sponsored-by: Catalyst IT
Comment 11 Alex Buckley 2020-09-24 11:40:39 UTC
Thanks David and Katrin. 

Attached is a unit test patch amending t/db_dependent/OAI/Sets.t to cover changes to C4/OAI/Sets.t

Could you please re-test?

Many thanks,
Alex
Comment 12 David Nind 2020-09-24 21:15:36 UTC
Hi Alex.

I retested, but the tests failed for me:

root@kohadevbox:koha(bz25460)$ prove t/db_dependent/OAI/Sets.t
t/db_dependent/OAI/Sets.t .. 1/153 C4::OAI::Sets::AddOAISetsBiblios(): DBI Exception: DBD::mysql::st execute failed: Duplicate entry '444-7' for key 'PRIMARY' [for Statement "
        INSERT INTO oai_sets_biblios (set_id, biblionumber)
        VALUES (?,?)
    " with ParamValues: 0='7', 1=444] at t/db_dependent/OAI/Sets.t line 370
# Looks like your test exited with 255 just after 100.
t/db_dependent/OAI/Sets.t .. Dubious, test returned 255 (wstat 65280, 0xff00)
Failed 53/153 subtests 

Test Summary Report
-------------------
t/db_dependent/OAI/Sets.t (Wstat: 65280 Tests: 100 Failed: 0)
  Non-zero exit status: 255
  Parse errors: Bad plan.  You planned 153 tests but ran 100.
Files=1, Tests=100,  2 wallclock secs ( 0.03 usr  0.01 sys +  1.50 cusr  0.15 csys =  1.69 CPU)
Result: FAIL

Note:
1. If I apply the patch on a clean setup and don't enable the OAI-PMH system preferences the tests pass.
2. If I apply the patch on a clean setup and enable the OAI-PMH system preferences the test fail.

Also, a really minor thing - there is an unnecessary " at the end of the description note for the new system preference.
Comment 13 Alex Buckley 2020-09-27 20:42:36 UTC
Hi David, 

Ohh, thanks for noting that! I'll look into why that's happening and hopefully update the patches on here today.

Thanks,
Alex
Comment 14 Alex Buckley 2020-09-28 11:08:02 UTC
Just noting I've found this problem appears on master too. When you have the  OAI-PMH:AutoUpdateSets  syspref disabled the t/db_dependent/OAI/Sets.t tests pass, and when that syspref is enabled they fail. Thanks for noting it David! 

I'll work on a fix for this.
Comment 15 Alex Buckley 2020-10-01 23:40:51 UTC
Created attachment 111051 [details] [review]
Bug 25460 : Add unit test for change to C4::OAI::Sets->UpdateOAISetsBiblio()

Test plan:

1. Run unit test:
sudo koha-shell <instancename>
prove t/db_dependent/OAI/Sets.t

2. All tests should pass

Sponsored-by: Catalyst IT
Comment 16 Alex Buckley 2020-10-01 23:42:16 UTC
Hi David, 

I've written a fix for the failing t/db_dependent/OAI/Sets.t unit tests when the OAI-PMH:AutoUpdateSets syspref is enabled. 

As noted in comment #14 this unit test is failing on master when that syspref is enabled, so as this problem is separate from this enhancement I've put the fix on a separate bug report bug 26589

Could you please re-test this patchset after applying 26589 first?

Many thanks,
Alex
Comment 17 David Nind 2020-10-02 10:03:39 UTC
Created attachment 111085 [details] [review]
Bug 25460: Update OAI sets when adding/editing/deleting item records

Also adds syspref to optionally choose to embed item information in record when adding/editing bibliographic and item records.

Test plan:
1. Enable OAI-PMH and OAI-PMH:AutoUpdateSets sysprefs
2. Create a biblio record with 1 item with CCODE of NFIC
3. In Administration > OAI Sets configuration create a set and define
the mapping:
Field: 952, Subfield: 8, Operator: is equal to,  Value: FIC

4. Run the build_oai_sets.pl script:
 cd misc/migration_tools
 sudo koha-shell <instancename>
 ./build_oai_sets.pl -i

5. Check what items are in the OAI set:
sudo koha-mysql <instancename>
select * from oai_sets_biblios;

6. Observe your bib from step 2 is not in the set
7. [ITEM EDIT] Edit the item to have CCODE of FIC repeat step 5 and observe biblio is still not in OAI set
8. [ITEM ADD] Add a new item with CCODE of FIC and repeat step 5 and observe biblio
still not in OAI set
9. [ITEM DELETE] Delete one of your 2 items and repeat step 5 and observe biblio still
not in OAI set

10. Apply patch, update database and restart plack (instruction below on
the latter 2 actions):

cd installer/data/mysql
sudo koha-shell <instancename>
./updatedatabase.pl
sudo koha-plack --restart <instancename>

11. Enable new syspref OAI-PMH:AutoUpdateSetsEmbedItemData
12. [ITEM ADD] Repeat step 8 and observe biblio is now in the OAI set
13. [ITEM EDIT] Edit both items to have a CCODE of NFIC and repeat step 5 and
observe bib now not in set
14. [ITEM EDIT] Edit one of the items to have CCODE of FIC and repeat step 5
and observe bib now in set
15. [ITEM DELETE] Delete all items off bib and repeat step 5 and observe bib
no longer in set

Sponsored-By: Catalyst IT

Signed-off-by: David Nind <david@davidnind.com>
Comment 18 David Nind 2020-10-02 10:03:49 UTC
Created attachment 111086 [details] [review]
Bug 25460 : Add unit test for change to C4::OAI::Sets->UpdateOAISetsBiblio()

Test plan:

1. Run unit test:
sudo koha-shell <instancename>
prove t/db_dependent/OAI/Sets.t

2. All tests should pass

Sponsored-by: Catalyst IT

Signed-off-by: David Nind <david@davidnind.com>
Comment 19 Katrin Fischer 2020-10-14 22:02:34 UTC
Created attachment 111644 [details] [review]
Bug 25460: Update OAI sets when adding/editing/deleting item records

Also adds syspref to optionally choose to embed item information in record when adding/editing bibliographic and item records.

Test plan:
1. Enable OAI-PMH and OAI-PMH:AutoUpdateSets sysprefs
2. Create a biblio record with 1 item with CCODE of NFIC
3. In Administration > OAI Sets configuration create a set and define
the mapping:
Field: 952, Subfield: 8, Operator: is equal to,  Value: FIC

4. Run the build_oai_sets.pl script:
 cd misc/migration_tools
 sudo koha-shell <instancename>
 ./build_oai_sets.pl -i

5. Check what items are in the OAI set:
sudo koha-mysql <instancename>
select * from oai_sets_biblios;

6. Observe your bib from step 2 is not in the set
7. [ITEM EDIT] Edit the item to have CCODE of FIC repeat step 5 and observe biblio is still not in OAI set
8. [ITEM ADD] Add a new item with CCODE of FIC and repeat step 5 and observe biblio
still not in OAI set
9. [ITEM DELETE] Delete one of your 2 items and repeat step 5 and observe biblio still
not in OAI set

10. Apply patch, update database and restart plack (instruction below on
the latter 2 actions):

cd installer/data/mysql
sudo koha-shell <instancename>
./updatedatabase.pl
sudo koha-plack --restart <instancename>

11. Enable new syspref OAI-PMH:AutoUpdateSetsEmbedItemData
12. [ITEM ADD] Repeat step 8 and observe biblio is now in the OAI set
13. [ITEM EDIT] Edit both items to have a CCODE of NFIC and repeat step 5 and
observe bib now not in set
14. [ITEM EDIT] Edit one of the items to have CCODE of FIC and repeat step 5
and observe bib now in set
15. [ITEM DELETE] Delete all items off bib and repeat step 5 and observe bib
no longer in set

Sponsored-By: Catalyst IT

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Comment 20 Katrin Fischer 2020-10-14 22:02:39 UTC
Created attachment 111645 [details] [review]
Bug 25460: Add unit test for change to C4::OAI::Sets->UpdateOAISetsBiblio()

Test plan:

1. Run unit test:
sudo koha-shell <instancename>
prove t/db_dependent/OAI/Sets.t

2. All tests should pass

Sponsored-by: Catalyst IT

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Comment 21 Katrin Fischer 2020-10-14 22:02:44 UTC
Created attachment 111646 [details] [review]
Bug 25460: (QA follow-up) Readd ending punctuation to sys pref description

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Comment 22 Alex Buckley 2020-10-14 22:06:15 UTC
Thank you David and Katrin!
Comment 23 Katrin Fischer 2020-10-14 22:08:52 UTC
I wonder if the bug description might be a little misleading. 

This makes it possible to use 952 in your OAI set mappings and it makes the updates of the OAI sets happen nicely when items are changed/added/updated.

But what it not does is embedding item information in the output formats - you still have to set the conf file and flag for marcxml in order to make that happen. So I am changing the title slightly, let me know if you disagree.
Comment 24 Katrin Fischer 2020-10-14 22:09:50 UTC
Created attachment 111647 [details] [review]
Bug 25460: (QA follow-up) Remove stray quote from system preference descrpition
Comment 25 Alex Buckley 2020-10-14 22:51:16 UTC
Hi Katrin, 

Thank you for clarifying that. I agree with your changes. 

This patchset ensures OAI sets are altered when items are added/altered (if 952 subfields are defined in the OAI set mappings). But as you say, it doesn't mean item data will automatically be embedded in the OAI sets output.

Thanks,
Alex
Comment 26 Jonathan Druart 2020-10-15 10:05:59 UTC
Created attachment 111694 [details] [review]
Bug 25460: Add unit test for change to C4::OAI::Sets->UpdateOAISetsBiblio()

Test plan:

1. Run unit test:
sudo koha-shell <instancename>
prove t/db_dependent/OAI/Sets.t

2. All tests should pass

Sponsored-by: Catalyst IT

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

JD amended patch: Put tests inside a subtest, and perltidy
Comment 27 Jonathan Druart 2020-10-15 10:08:09 UTC
Created attachment 111695 [details] [review]
Bug 25460: (QA follow-up) Remove stray quote from system preference description
Comment 28 Jonathan Druart 2020-10-15 10:55:12 UTC
Pushed to master for 20.11, thanks to everybody involved!
Comment 29 Jonathan Druart 2020-10-15 12:01:09 UTC
Created attachment 111715 [details] [review]
Bug 25460: Remove swp file

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>