Bug 15869 added the ability to specify frameworks when overlaying while importing, we should extend this to allow updating the framework for records imported from OCLC connexion
Created attachment 149203 [details] [review] Bug 33419: POC untested This will require testing, and better instructions on how to match etc. To test: 1 - Supply a value for 'framework' and 'overlay_framework' in the connxion.conf file 2 - Setup connexion, there is some help here on testing values: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=15720 3 - Test importing bibs that match, and that don't match 4 - Confirm the framework value is used for new records 5 - Confirm if framework is blank the default framework is used for new records 6 - Confirm if framework is not supplied the default framework is used for new records 7 - Confirm a matching record that overlays obeys the overlay_framework param 8 - Confirm overlayed record uses default framework if the param is blank 9 - Confirm the original records framework is retained if the param is not passed
TO DO: cronjobs/import_webservice_batch.pl
Created attachment 162338 [details] [review] Bug 33418: Add overlay_framework option to connexion scripts This patch adds the option to specify a framework to be used when overlaying records from webservices/connexion To test: 1 - vim /etc/koha/sites/kohadev/connexion.cnf 2 - Set content: host: port: 8888 koha:http://localhost:8081 log:/var/log/koha/kohadev/connexion.log match:ISBN user:kohauser password:kohapass overlay_action:replace nomatch_action:create_new item_action:always_add import_mode:direct framework:BKS overlay_framework: debug:1 3 - Save the sample file from this bug into your kohaclone (or copy and paste into a file your koha test site can reach) 4 - On the command line: perl misc/bin/connexion_import_daemon.pl -c /etc/koha/sites/kohadev/connexion.cnf 5 - In another terminal: cat bug_33418.test | nc -v localhost 8888 6 - It should report success and a biblionumber 7 - In Koha: Cataloguing->Manage staged MARC record 8 - View the most recent batch with file name (webservice) 9 - Confirm it was imported, no match 10 - Click 'View' under the 'Record' column 11 - Confirm record loads correctly and 'MARC framework' detail is 'Books, Booklets, Workbooks' 12 - On the terminal repeat the command: cat bug_33418.test | nc -v localhost 8888 13 - It should succed 14 - View the new batch, confirm the record matched this time 15 - View the record details, confirm framework is now 'default' 16 - On the first terminal hit Ctrl+C to stop the daemon 16 - Edit connexion.cnf and change: import_mode:stage framework:ACQ overlay_framework:IR 17 - Restart daemon: perl misc/bin/connexion_import_daemon.pl -c /etc/koha/sites/kohadev/connexion.cnf 18 - Delete the record created above 19 - On the second terminal repeat the command: cat bug_33418.test | nc -v localhost 8888 20 - Confirm the batch is created, but not imported 21 - In terminal: perl misc/cronjobs/import_webservice_batch.pl --framework=ACQ --overlay_framework=BKS 22 - Confirm batch imported, and record in ACQ framework 23 - In terminal: cat bug_33418.test | nc -v localhost 8888 perl misc/cronjobs/import_webservice_batch.pl --framework=ACQ --overlay_framework=BKS 24 - Confirm batch added, record matched, record imported, and record now in Books framework 25 - Stop the deamon, edit connexion.cnf: import_mode:direct 26 - Start the daemon, and on other terminal repeat: cat bug_33418.test | nc -v localhost 8888 27 - Confirm record in Binders framework 28 - Set record framework to Books 29 - Stop daemon, edit cnf and remove 'overlay_framework' setting 30 - Start daemon and cat the file again 31 - Confirm the record remains in Books framework
Created attachment 162339 [details] Test records for Connexion import
Created attachment 162471 [details] [review] Bug 33418: Add overlay_framework option to connexion scripts This patch adds the option to specify a framework to be used when overlaying records from webservices/connexion To test: 1 - vim /etc/koha/sites/kohadev/connexion.cnf 2 - Set content: host: port: 8888 koha:http://localhost:8081 log:/var/log/koha/kohadev/connexion.log match:ISBN user:kohauser password:kohapass overlay_action:replace nomatch_action:create_new item_action:always_add import_mode:direct framework:BKS overlay_framework: debug:1 3 - Save the sample file from this bug into your kohaclone (or copy and paste into a file your koha test site can reach) 4 - On the command line: perl misc/bin/connexion_import_daemon.pl -c /etc/koha/sites/kohadev/connexion.cnf 5 - In another terminal: cat bug_33418.test | nc -v localhost 8888 6 - It should report success and a biblionumber 7 - In Koha: Cataloguing->Manage staged MARC record 8 - View the most recent batch with file name (webservice) 9 - Confirm it was imported, no match 10 - Click 'View' under the 'Record' column 11 - Confirm record loads correctly and 'MARC framework' detail is 'Books, Booklets, Workbooks' 12 - On the terminal repeat the command: cat bug_33418.test | nc -v localhost 8888 13 - It should succed 14 - View the new batch, confirm the record matched this time 15 - View the record details, confirm framework is now 'default' 16 - On the first terminal hit Ctrl+C to stop the daemon 16 - Edit connexion.cnf and change: import_mode:stage framework:ACQ overlay_framework:IR 17 - Restart daemon: perl misc/bin/connexion_import_daemon.pl -c /etc/koha/sites/kohadev/connexion.cnf 18 - Delete the record created above 19 - On the second terminal repeat the command: cat bug_33418.test | nc -v localhost 8888 20 - Confirm the batch is created, but not imported 21 - In terminal: perl misc/cronjobs/import_webservice_batch.pl --framework=ACQ --overlay_framework=BKS 22 - Confirm batch imported, and record in ACQ framework 23 - In terminal: cat bug_33418.test | nc -v localhost 8888 perl misc/cronjobs/import_webservice_batch.pl --framework=ACQ --overlay_framework=BKS 24 - Confirm batch added, record matched, record imported, and record now in Books framework 25 - Stop the deamon, edit connexion.cnf: import_mode:direct 26 - Start the daemon, and on other terminal repeat: cat bug_33418.test | nc -v localhost 8888 27 - Confirm record in Binders framework 28 - Set record framework to Books 29 - Stop daemon, edit cnf and remove 'overlay_framework' setting 30 - Start daemon and cat the file again 31 - Confirm the record remains in Books framework Signed-off-by: Brendan Lawlor <blawlor@clamsnet.org>
Thanks to your detailed instructions it was fun testing this bug. Test notes: Update system prefs to allow weak passwords Set up kohauser with Cataloging permissions, thanks for link to bug Follow step 18 to delete the record to avoid confusion\ Sometimes when trying to restart the daemon I would get an error like perl misc/bin/connexion_import_daemon.pl -c /etc/koha/sites/kohadev/connexion.cnf Couldn't be a tcp server on port 8888: Address already in use IO::Socket::INET: Address already in use at misc/bin/connexion_import_daemon.pl line 209. but I found you can resolve this issue with sudo fuser 8888/tcp -k When I first tried the last step it overlaid the record with the default framework, but I realized that was because I did not remove the whole line overlay_framework. It took me a couple tries to realize I needed to remove the whole line, but I eventually got it.
Created attachment 166518 [details] [review] Bug 33418: Add overlay_framework option to connexion scripts This patch adds the option to specify a framework to be used when overlaying records from webservices/connexion To test: 1 - vim /etc/koha/sites/kohadev/connexion.cnf 2 - Set content: host: port: 8888 koha:http://localhost:8081 log:/var/log/koha/kohadev/connexion.log match:ISBN user:kohauser password:kohapass overlay_action:replace nomatch_action:create_new item_action:always_add import_mode:direct framework:BKS overlay_framework: debug:1 3 - Save the sample file from this bug into your kohaclone (or copy and paste into a file your koha test site can reach) 4 - On the command line: perl misc/bin/connexion_import_daemon.pl -c /etc/koha/sites/kohadev/connexion.cnf 5 - In another terminal: cat bug_33418.test | nc -v localhost 8888 6 - It should report success and a biblionumber 7 - In Koha: Cataloguing->Manage staged MARC record 8 - View the most recent batch with file name (webservice) 9 - Confirm it was imported, no match 10 - Click 'View' under the 'Record' column 11 - Confirm record loads correctly and 'MARC framework' detail is 'Books, Booklets, Workbooks' 12 - On the terminal repeat the command: cat bug_33418.test | nc -v localhost 8888 13 - It should succed 14 - View the new batch, confirm the record matched this time 15 - View the record details, confirm framework is now 'default' 16 - On the first terminal hit Ctrl+C to stop the daemon 16 - Edit connexion.cnf and change: import_mode:stage framework:ACQ overlay_framework:IR 17 - Restart daemon: perl misc/bin/connexion_import_daemon.pl -c /etc/koha/sites/kohadev/connexion.cnf 18 - Delete the record created above 19 - On the second terminal repeat the command: cat bug_33418.test | nc -v localhost 8888 20 - Confirm the batch is created, but not imported 21 - In terminal: perl misc/cronjobs/import_webservice_batch.pl --framework=ACQ --overlay_framework=BKS 22 - Confirm batch imported, and record in ACQ framework 23 - In terminal: cat bug_33418.test | nc -v localhost 8888 perl misc/cronjobs/import_webservice_batch.pl --framework=ACQ --overlay_framework=BKS 24 - Confirm batch added, record matched, record imported, and record now in Books framework 25 - Stop the deamon, edit connexion.cnf: import_mode:direct 26 - Start the daemon, and on other terminal repeat: cat bug_33418.test | nc -v localhost 8888 27 - Confirm record in Binders framework 28 - Set record framework to Books 29 - Stop daemon, edit cnf and remove 'overlay_framework' setting 30 - Start daemon and cat the file again 31 - Confirm the record remains in Books framework Signed-off-by: Brendan Lawlor <blawlor@clamsnet.org> Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Essentially only adding the overlay framework parameter. Not tested but patch looks good to me.
Pushed for 24.05! Well done everyone, thank you!
Not backported to 23.11.x