Description
Alex Buckley
2020-05-12 02:41:23 UTC
Created attachment 104721 [details] [review] Bug 25460: Update OAI sets when adding/editing/deleting item records Test plan: 1. Enable OAI-PMH and OAI-PMH:AutoUpdateSets sysprefs 2. Create a biblio record with 1 item with CCODE of NFIC 3. In Administration > OAI Sets configuration create a set and define the mapping: Field: 952, Subfield: 8, Operator: is equal to, Value: FIC 4. Run the build_oai_sets.pl script: cd misc/migration_tools sudo koha-shell <instancename> ./build_oai_sets.pl -i 5. Check what items are in the OAI set: sudo koha-mysql <instancename> select * from oai_sets_biblios; 6. Observe your bib from step 2 is not in the set 7. [EDIT] Edit the item to have CCODE of FIC repeat step 5 and observe biblio is still not in OAI set 8. [ADD] Add a new item with CCODE of FIC and repeat step 5 and observe biblio still not in OAI set 9. [DELETE] Delete one of your 2 items and repeat step 5 and observe biblio still not in OAI set 10. Apply patch, update database and restart plack (instruction below on the latter 2 actions): cd installer/data/mysql sudo koha-shell <instancename> ./updatedatabase.pl sudo koha-plack --restart <instancename> 11. Enable new syspref OAI-PMH:AutoUpdateSetsEmbedItemData 12. [ADD] Repeat step 8 and observe biblio is now in the OAI set 13. [EDIT] Edit both items to have a CCODE of NFIC and repeat step 5 and observe bib now not in set 14. [EDIT] Edit one of the items to have CCODE of FIC and repeat step 5 and observe bib now in set 15. [DELETE] Delete all items off bib and repeat step 5 and observe bib no longer in set Sponsored-By: Catalyst IT Created attachment 104983 [details] [review] Bug 25460: Update OAI sets when adding/editing/deleting item records Test plan: 1. Enable OAI-PMH and OAI-PMH:AutoUpdateSets sysprefs 2. Create a biblio record with 1 item with CCODE of NFIC 3. In Administration > OAI Sets configuration create a set and define the mapping: Field: 952, Subfield: 8, Operator: is equal to, Value: FIC 4. Run the build_oai_sets.pl script: cd misc/migration_tools sudo koha-shell <instancename> ./build_oai_sets.pl -i 5. Check what items are in the OAI set: sudo koha-mysql <instancename> select * from oai_sets_biblios; 6. Observe your bib from step 2 is not in the set 7. [EDIT] Edit the item to have CCODE of FIC repeat step 5 and observe biblio is still not in OAI set 8. [ADD] Add a new item with CCODE of FIC and repeat step 5 and observe biblio still not in OAI set 9. [DELETE] Delete one of your 2 items and repeat step 5 and observe biblio still not in OAI set 10. Apply patch, update database and restart plack (instruction below on the latter 2 actions): cd installer/data/mysql sudo koha-shell <instancename> ./updatedatabase.pl sudo koha-plack --restart <instancename> 11. Enable new syspref OAI-PMH:AutoUpdateSetsEmbedItemData 12. [ADD] Repeat step 8 and observe biblio is now in the OAI set 13. [EDIT] Edit both items to have a CCODE of NFIC and repeat step 5 and observe bib now not in set 14. [EDIT] Edit one of the items to have CCODE of FIC and repeat step 5 and observe bib now in set 15. [DELETE] Delete all items off bib and repeat step 5 and observe bib no longer in set Sponsored-By: Catalyst IT Created attachment 104985 [details] [review] Bug 25460: Update OAI sets when adding/editing/deleting item records Test plan: 1. Enable OAI-PMH and OAI-PMH:AutoUpdateSets sysprefs 2. Create a biblio record with 1 item with CCODE of NFIC 3. In Administration > OAI Sets configuration create a set and define the mapping: Field: 952, Subfield: 8, Operator: is equal to, Value: FIC 4. Run the build_oai_sets.pl script: cd misc/migration_tools sudo koha-shell <instancename> ./build_oai_sets.pl -i 5. Check what items are in the OAI set: sudo koha-mysql <instancename> select * from oai_sets_biblios; 6. Observe your bib from step 2 is not in the set 7. [EDIT] Edit the item to have CCODE of FIC repeat step 5 and observe biblio is still not in OAI set 8. [ADD] Add a new item with CCODE of FIC and repeat step 5 and observe biblio still not in OAI set 9. [DELETE] Delete one of your 2 items and repeat step 5 and observe biblio still not in OAI set 10. Apply patch, update database and restart plack (instruction below on the latter 2 actions): cd installer/data/mysql sudo koha-shell <instancename> ./updatedatabase.pl sudo koha-plack --restart <instancename> 11. Enable new syspref OAI-PMH:AutoUpdateSetsEmbedItemData 12. [ADD] Repeat step 8 and observe biblio is now in the OAI set 13. [EDIT] Edit both items to have a CCODE of NFIC and repeat step 5 and observe bib now not in set 14. [EDIT] Edit one of the items to have CCODE of FIC and repeat step 5 and observe bib now in set 15. [DELETE] Delete all items off bib and repeat step 5 and observe bib no longer in set Sponsored-By: Catalyst IT Created attachment 104986 [details] [review] Bug 25460: Update OAI sets when adding/editing/deleting item records Test plan: 1. Enable OAI-PMH and OAI-PMH:AutoUpdateSets sysprefs 2. Create a biblio record with 1 item with CCODE of NFIC 3. In Administration > OAI Sets configuration create a set and define the mapping: Field: 952, Subfield: 8, Operator: is equal to, Value: FIC 4. Run the build_oai_sets.pl script: cd misc/migration_tools sudo koha-shell <instancename> ./build_oai_sets.pl -i 5. Check what items are in the OAI set: sudo koha-mysql <instancename> select * from oai_sets_biblios; 6. Observe your bib from step 2 is not in the set 7. [EDIT] Edit the item to have CCODE of FIC repeat step 5 and observe biblio is still not in OAI set 8. [ADD] Add a new item with CCODE of FIC and repeat step 5 and observe biblio still not in OAI set 9. [DELETE] Delete one of your 2 items and repeat step 5 and observe biblio still not in OAI set 10. Apply patch, update database and restart plack (instruction below on the latter 2 actions): cd installer/data/mysql sudo koha-shell <instancename> ./updatedatabase.pl sudo koha-plack --restart <instancename> 11. Enable new syspref OAI-PMH:AutoUpdateSetsEmbedItemData 12. [ADD] Repeat step 8 and observe biblio is now in the OAI set 13. [EDIT] Edit both items to have a CCODE of NFIC and repeat step 5 and observe bib now not in set 14. [EDIT] Edit one of the items to have CCODE of FIC and repeat step 5 and observe bib now in set 15. [DELETE] Delete all items off bib and repeat step 5 and observe bib no longer in set Sponsored-By: Catalyst IT Created attachment 104987 [details] [review] Bug 25460: Update OAI sets when adding/editing/deleting item records Also adds syspref to optionally choose to embed item information in record when adding/editing bibliographic and item records. Test plan: 1. Enable OAI-PMH and OAI-PMH:AutoUpdateSets sysprefs 2. Create a biblio record with 1 item with CCODE of NFIC 3. In Administration > OAI Sets configuration create a set and define the mapping: Field: 952, Subfield: 8, Operator: is equal to, Value: FIC 4. Run the build_oai_sets.pl script: cd misc/migration_tools sudo koha-shell <instancename> ./build_oai_sets.pl -i 5. Check what items are in the OAI set: sudo koha-mysql <instancename> select * from oai_sets_biblios; 6. Observe your bib from step 2 is not in the set 7. [ITEM EDIT] Edit the item to have CCODE of FIC repeat step 5 and observe biblio is still not in OAI set 8. [ITEM ADD] Add a new item with CCODE of FIC and repeat step 5 and observe biblio still not in OAI set 9. [ITEM DELETE] Delete one of your 2 items and repeat step 5 and observe biblio still not in OAI set 10. Apply patch, update database and restart plack (instruction below on the latter 2 actions): cd installer/data/mysql sudo koha-shell <instancename> ./updatedatabase.pl sudo koha-plack --restart <instancename> 11. Enable new syspref OAI-PMH:AutoUpdateSetsEmbedItemData 12. [ITEM ADD] Repeat step 8 and observe biblio is now in the OAI set 13. [ITEM EDIT] Edit both items to have a CCODE of NFIC and repeat step 5 and observe bib now not in set 14. [ITEM EDIT] Edit one of the items to have CCODE of FIC and repeat step 5 and observe bib now in set 15. [ITEM DELETE] Delete all items off bib and repeat step 5 and observe bib no longer in set Sponsored-By: Catalyst IT Created attachment 109560 [details] [review] Bug 25460: Update OAI sets when adding/editing/deleting item records Also adds syspref to optionally choose to embed item information in record when adding/editing bibliographic and item records. Test plan: 1. Enable OAI-PMH and OAI-PMH:AutoUpdateSets sysprefs 2. Create a biblio record with 1 item with CCODE of NFIC 3. In Administration > OAI Sets configuration create a set and define the mapping: Field: 952, Subfield: 8, Operator: is equal to, Value: FIC 4. Run the build_oai_sets.pl script: cd misc/migration_tools sudo koha-shell <instancename> ./build_oai_sets.pl -i 5. Check what items are in the OAI set: sudo koha-mysql <instancename> select * from oai_sets_biblios; 6. Observe your bib from step 2 is not in the set 7. [ITEM EDIT] Edit the item to have CCODE of FIC repeat step 5 and observe biblio is still not in OAI set 8. [ITEM ADD] Add a new item with CCODE of FIC and repeat step 5 and observe biblio still not in OAI set 9. [ITEM DELETE] Delete one of your 2 items and repeat step 5 and observe biblio still not in OAI set 10. Apply patch, update database and restart plack (instruction below on the latter 2 actions): cd installer/data/mysql sudo koha-shell <instancename> ./updatedatabase.pl sudo koha-plack --restart <instancename> 11. Enable new syspref OAI-PMH:AutoUpdateSetsEmbedItemData 12. [ITEM ADD] Repeat step 8 and observe biblio is now in the OAI set 13. [ITEM EDIT] Edit both items to have a CCODE of NFIC and repeat step 5 and observe bib now not in set 14. [ITEM EDIT] Edit one of the items to have CCODE of FIC and repeat step 5 and observe bib now in set 15. [ITEM DELETE] Delete all items off bib and repeat step 5 and observe bib no longer in set Sponsored-By: Catalyst IT Rebased against master Created attachment 110435 [details] [review] Bug 25460: Update OAI sets when adding/editing/deleting item records Also adds syspref to optionally choose to embed item information in record when adding/editing bibliographic and item records. Test plan: 1. Enable OAI-PMH and OAI-PMH:AutoUpdateSets sysprefs 2. Create a biblio record with 1 item with CCODE of NFIC 3. In Administration > OAI Sets configuration create a set and define the mapping: Field: 952, Subfield: 8, Operator: is equal to, Value: FIC 4. Run the build_oai_sets.pl script: cd misc/migration_tools sudo koha-shell <instancename> ./build_oai_sets.pl -i 5. Check what items are in the OAI set: sudo koha-mysql <instancename> select * from oai_sets_biblios; 6. Observe your bib from step 2 is not in the set 7. [ITEM EDIT] Edit the item to have CCODE of FIC repeat step 5 and observe biblio is still not in OAI set 8. [ITEM ADD] Add a new item with CCODE of FIC and repeat step 5 and observe biblio still not in OAI set 9. [ITEM DELETE] Delete one of your 2 items and repeat step 5 and observe biblio still not in OAI set 10. Apply patch, update database and restart plack (instruction below on the latter 2 actions): cd installer/data/mysql sudo koha-shell <instancename> ./updatedatabase.pl sudo koha-plack --restart <instancename> 11. Enable new syspref OAI-PMH:AutoUpdateSetsEmbedItemData 12. [ITEM ADD] Repeat step 8 and observe biblio is now in the OAI set 13. [ITEM EDIT] Edit both items to have a CCODE of NFIC and repeat step 5 and observe bib now not in set 14. [ITEM EDIT] Edit one of the items to have CCODE of FIC and repeat step 5 and observe bib now in set 15. [ITEM DELETE] Delete all items off bib and repeat step 5 and observe bib no longer in set Sponsored-By: Catalyst IT Signed-off-by: David Nind <david@davidnind.com> Hi Alex, can we please have a unit test highlighting the change? Created attachment 110645 [details] [review] Bug 25460 : Add unit test for change to C4::OAI::Sets->UpdateOAISetsBiblio() Test plan: 1. Run unit test: sudo koha-shell <instancename> prove t/db_dependent/OAI/Sets.t 2. All tests should pass Sponsored-by: Catalyst IT Thanks David and Katrin. Attached is a unit test patch amending t/db_dependent/OAI/Sets.t to cover changes to C4/OAI/Sets.t Could you please re-test? Many thanks, Alex Hi Alex. I retested, but the tests failed for me: root@kohadevbox:koha(bz25460)$ prove t/db_dependent/OAI/Sets.t t/db_dependent/OAI/Sets.t .. 1/153 C4::OAI::Sets::AddOAISetsBiblios(): DBI Exception: DBD::mysql::st execute failed: Duplicate entry '444-7' for key 'PRIMARY' [for Statement " INSERT INTO oai_sets_biblios (set_id, biblionumber) VALUES (?,?) " with ParamValues: 0='7', 1=444] at t/db_dependent/OAI/Sets.t line 370 # Looks like your test exited with 255 just after 100. t/db_dependent/OAI/Sets.t .. Dubious, test returned 255 (wstat 65280, 0xff00) Failed 53/153 subtests Test Summary Report ------------------- t/db_dependent/OAI/Sets.t (Wstat: 65280 Tests: 100 Failed: 0) Non-zero exit status: 255 Parse errors: Bad plan. You planned 153 tests but ran 100. Files=1, Tests=100, 2 wallclock secs ( 0.03 usr 0.01 sys + 1.50 cusr 0.15 csys = 1.69 CPU) Result: FAIL Note: 1. If I apply the patch on a clean setup and don't enable the OAI-PMH system preferences the tests pass. 2. If I apply the patch on a clean setup and enable the OAI-PMH system preferences the test fail. Also, a really minor thing - there is an unnecessary " at the end of the description note for the new system preference. Hi David, Ohh, thanks for noting that! I'll look into why that's happening and hopefully update the patches on here today. Thanks, Alex Just noting I've found this problem appears on master too. When you have the OAI-PMH:AutoUpdateSets syspref disabled the t/db_dependent/OAI/Sets.t tests pass, and when that syspref is enabled they fail. Thanks for noting it David! I'll work on a fix for this. Created attachment 111051 [details] [review] Bug 25460 : Add unit test for change to C4::OAI::Sets->UpdateOAISetsBiblio() Test plan: 1. Run unit test: sudo koha-shell <instancename> prove t/db_dependent/OAI/Sets.t 2. All tests should pass Sponsored-by: Catalyst IT Hi David, I've written a fix for the failing t/db_dependent/OAI/Sets.t unit tests when the OAI-PMH:AutoUpdateSets syspref is enabled. As noted in comment #14 this unit test is failing on master when that syspref is enabled, so as this problem is separate from this enhancement I've put the fix on a separate bug report bug 26589 Could you please re-test this patchset after applying 26589 first? Many thanks, Alex Created attachment 111085 [details] [review] Bug 25460: Update OAI sets when adding/editing/deleting item records Also adds syspref to optionally choose to embed item information in record when adding/editing bibliographic and item records. Test plan: 1. Enable OAI-PMH and OAI-PMH:AutoUpdateSets sysprefs 2. Create a biblio record with 1 item with CCODE of NFIC 3. In Administration > OAI Sets configuration create a set and define the mapping: Field: 952, Subfield: 8, Operator: is equal to, Value: FIC 4. Run the build_oai_sets.pl script: cd misc/migration_tools sudo koha-shell <instancename> ./build_oai_sets.pl -i 5. Check what items are in the OAI set: sudo koha-mysql <instancename> select * from oai_sets_biblios; 6. Observe your bib from step 2 is not in the set 7. [ITEM EDIT] Edit the item to have CCODE of FIC repeat step 5 and observe biblio is still not in OAI set 8. [ITEM ADD] Add a new item with CCODE of FIC and repeat step 5 and observe biblio still not in OAI set 9. [ITEM DELETE] Delete one of your 2 items and repeat step 5 and observe biblio still not in OAI set 10. Apply patch, update database and restart plack (instruction below on the latter 2 actions): cd installer/data/mysql sudo koha-shell <instancename> ./updatedatabase.pl sudo koha-plack --restart <instancename> 11. Enable new syspref OAI-PMH:AutoUpdateSetsEmbedItemData 12. [ITEM ADD] Repeat step 8 and observe biblio is now in the OAI set 13. [ITEM EDIT] Edit both items to have a CCODE of NFIC and repeat step 5 and observe bib now not in set 14. [ITEM EDIT] Edit one of the items to have CCODE of FIC and repeat step 5 and observe bib now in set 15. [ITEM DELETE] Delete all items off bib and repeat step 5 and observe bib no longer in set Sponsored-By: Catalyst IT Signed-off-by: David Nind <david@davidnind.com> Created attachment 111086 [details] [review] Bug 25460 : Add unit test for change to C4::OAI::Sets->UpdateOAISetsBiblio() Test plan: 1. Run unit test: sudo koha-shell <instancename> prove t/db_dependent/OAI/Sets.t 2. All tests should pass Sponsored-by: Catalyst IT Signed-off-by: David Nind <david@davidnind.com> Created attachment 111644 [details] [review] Bug 25460: Update OAI sets when adding/editing/deleting item records Also adds syspref to optionally choose to embed item information in record when adding/editing bibliographic and item records. Test plan: 1. Enable OAI-PMH and OAI-PMH:AutoUpdateSets sysprefs 2. Create a biblio record with 1 item with CCODE of NFIC 3. In Administration > OAI Sets configuration create a set and define the mapping: Field: 952, Subfield: 8, Operator: is equal to, Value: FIC 4. Run the build_oai_sets.pl script: cd misc/migration_tools sudo koha-shell <instancename> ./build_oai_sets.pl -i 5. Check what items are in the OAI set: sudo koha-mysql <instancename> select * from oai_sets_biblios; 6. Observe your bib from step 2 is not in the set 7. [ITEM EDIT] Edit the item to have CCODE of FIC repeat step 5 and observe biblio is still not in OAI set 8. [ITEM ADD] Add a new item with CCODE of FIC and repeat step 5 and observe biblio still not in OAI set 9. [ITEM DELETE] Delete one of your 2 items and repeat step 5 and observe biblio still not in OAI set 10. Apply patch, update database and restart plack (instruction below on the latter 2 actions): cd installer/data/mysql sudo koha-shell <instancename> ./updatedatabase.pl sudo koha-plack --restart <instancename> 11. Enable new syspref OAI-PMH:AutoUpdateSetsEmbedItemData 12. [ITEM ADD] Repeat step 8 and observe biblio is now in the OAI set 13. [ITEM EDIT] Edit both items to have a CCODE of NFIC and repeat step 5 and observe bib now not in set 14. [ITEM EDIT] Edit one of the items to have CCODE of FIC and repeat step 5 and observe bib now in set 15. [ITEM DELETE] Delete all items off bib and repeat step 5 and observe bib no longer in set Sponsored-By: Catalyst IT Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de> Created attachment 111645 [details] [review] Bug 25460: Add unit test for change to C4::OAI::Sets->UpdateOAISetsBiblio() Test plan: 1. Run unit test: sudo koha-shell <instancename> prove t/db_dependent/OAI/Sets.t 2. All tests should pass Sponsored-by: Catalyst IT Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de> Created attachment 111646 [details] [review] Bug 25460: (QA follow-up) Readd ending punctuation to sys pref description Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de> Thank you David and Katrin! I wonder if the bug description might be a little misleading. This makes it possible to use 952 in your OAI set mappings and it makes the updates of the OAI sets happen nicely when items are changed/added/updated. But what it not does is embedding item information in the output formats - you still have to set the conf file and flag for marcxml in order to make that happen. So I am changing the title slightly, let me know if you disagree. Created attachment 111647 [details] [review] Bug 25460: (QA follow-up) Remove stray quote from system preference descrpition Hi Katrin, Thank you for clarifying that. I agree with your changes. This patchset ensures OAI sets are altered when items are added/altered (if 952 subfields are defined in the OAI set mappings). But as you say, it doesn't mean item data will automatically be embedded in the OAI sets output. Thanks, Alex Created attachment 111694 [details] [review] Bug 25460: Add unit test for change to C4::OAI::Sets->UpdateOAISetsBiblio() Test plan: 1. Run unit test: sudo koha-shell <instancename> prove t/db_dependent/OAI/Sets.t 2. All tests should pass Sponsored-by: Catalyst IT Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de> JD amended patch: Put tests inside a subtest, and perltidy Created attachment 111695 [details] [review] Bug 25460: (QA follow-up) Remove stray quote from system preference description Pushed to master for 20.11, thanks to everybody involved! Created attachment 111715 [details] [review] Bug 25460: Remove swp file Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org> enhancement will not be backported to 20.05.x |