Summary: | New Perl Class (Koha::ExternalContent::OCLC) for integrating with OCLC's APIs | ||
---|---|---|---|
Product: | Koha | Reporter: | Andreas Roussos <a.roussos> |
Component: | MARC Authority data support | Assignee: | Andreas Roussos <a.roussos> |
Status: | Needs Signoff --- | QA Contact: | Testopia <testopia> |
Severity: | enhancement | ||
Priority: | P5 - low | ||
Version: | Main | ||
Hardware: | All | ||
OS: | All | ||
GIT URL: | Change sponsored?: | --- | |
Patch complexity: | Medium patch | Documentation contact: | |
Documentation submission: | Text to go in the release notes: | ||
Version(s) released in: | Circulation function: | ||
Bug Depends on: | |||
Bug Blocks: | 40309, 40310, 40311 | ||
Attachments: |
Sample authority records (MARC21 flavour)
Sample authority records (UNIMARC flavour) Bug 40308: add new Perl Class Koha::ExternalContent::OCLC Bug 40308: add new System Preferences for OCLC Dewey API Bug 40308: add a command-line tool that updates authorities en masse |
Description
Andreas Roussos
2025-07-06 21:01:58 UTC
Created attachment 183809 [details]
Sample authority records (MARC21 flavour)
Created attachment 183810 [details]
Sample authority records (UNIMARC flavour)
Created attachment 183811 [details] [review] Bug 40308: add new Perl Class Koha::ExternalContent::OCLC OCLC are offering a number of web APIs, this is an attempt to collect utility functions in a new Perl Class. At the moment, the Class contains the following functions, that allow interaction with the OCLC Dewey Linked Data API: - get_access_token - get_ddc_uri - get_ddc_url - get_ddc_description Test plan: 1) Inside a KTD instance (MARC flavour does not play a role), run `perldoc -o html Koha::ExternalContent::OCLC > OCLC.html` and view the documentation for this class (OCLC.html) in your web browser. Ensure everything makes sense ;-) Created attachment 183812 [details] [review] Bug 40308: add new System Preferences for OCLC Dewey API This patch adds four new System Preferences that control the integration with the OCLC Dewey Linked Data API, namely: - OCLCAPIWSKeyClientID: Stores the client ID component of your OCLC API WSKey - OCLCAPIWSKeyClientSecret: Stores the client secret component of your OCLC API WSKey - OCLCDeweyLinkedDataAPI: Basically an On/Off "switch" for the lookup service - OCLCDeweyLinkedDataLanguage: Sets the language of the DDC description to be fetched Test plan: 0) Launch either a MARC21 or a UNIMARC KTD container (MARC flavour does not play a role for this particular test plan), then apply this patch. 1) Run `reset_all` to force your KTD instance to load installer/data/mysql/mandatory/sysprefs.sql. 2) Then, visit Administration > System preferences > Web services and inspect the four new sysprefs under 'OCLC Dewey Linked Data API'. Look for typos in the descriptions, etc. 3) Then, run `updatedatabase` to test the atomic update. It should complete without warnings/errors. 4) Finally, visually inspect the output of `git log -p C4/UsageStats.pm` for errors. Notice that the System Preferences pertaining to the WSKey client ID and secret were *not* added for security reasons. Created attachment 183813 [details] [review] Bug 40308: add a command-line tool that updates authorities en masse In Bugs 40309, 40310, and 40311 we will add three new Authority Framework plugins that allow for easy DDC lookups using OCLC's Dewey Linked Data API while cataloguing authorities in the Staff Interface. However, users may want to retroactively enrich their existing authorities using a command-line tool that can modify authorities en masse. This patch adds misc/maintenance/fetch_oclc_dewey_linked_data.pl, which is a Perl script that can accept a number of options in order to facilitate selective operation on specific authority types, or even authid ranges. Test plan (NOTE: you need to have an organizational account with OCLC and an active WebDewey subscription associated with it in order to request a WSKey for accessing the Dewey Linked Data API): [ I recommend working with two KTD instances in parallel. ] 0) First of all, run: ktd_proxy --start 1) Then, edit your KTD .env file so that: KOHA_MARC_FLAVOUR=marc21 Then: ktd --proxy --name marc21 up Wait until it starts up. Then, in a different terminal, edit your KTD .env file so that: KOHA_MARC_FLAVOUR=unimarc Then: ktd --proxy --name unimarc up Wait until it starts up. You now have two KTDs running, one for each MARC flavour you want to test. 2) Get inside the MARC21 Koha container: ktd --name marc21 --shell Download the sample MARC21 authority records (sample_auth_records_marc21.mrc) attached to this Bug report (the file contains 15 records): wget -O sample_auth_records_marc21.mrc https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=183809 Then, import the authorities: misc/migration_tools/bulkmarcimport.pl -a --file sample_auth_records_marc21.mrc -v -c MARC21 -m ISO2709 Repeat for the UNIMARC Koha container: ktd --name unimarc --shell Download the sample UNIMARC authority records (sample_auth_records_unimarc.mrc) attached to this Bug report (the file contains 12 records): wget -O sample_auth_records_unimarc.mrc https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=183810 Then, import the authorities: misc/migration_tools/bulkmarcimport.pl -a --file sample_auth_records_unimarc.mrc -v -c UNIMARC -m ISO2709 3) Once the sample authorities have been imported, you must fill in these two System Preferences (this is the tricky part, as, in order to request a WSKey you need to have an organizational account with OCLC and an active WebDewey subscription associated with it): - OCLCAPIWSKeyClientID - OCLCAPIWSKeyClientSecret 4) When the above System Preferences have been filled in, try running the new script (passing as many variations of the command-line options as you can). For example: cd misc/maintenance/ ./fetch_oclc_dewey_linked_data.pl -h ./fetch_oclc_dewey_linked_data.pl --man | cat ./fetch_oclc_dewey_linked_data.pl -v ./fetch_oclc_dewey_linked_data.pl -v -l el ./fetch_oclc_dewey_linked_data.pl -v -a SNC ./fetch_oclc_dewey_linked_data.pl -v -a SNC -w "authid = 5955" ./fetch_oclc_dewey_linked_data.pl -v -a SNC -w "authid = 5955" -c ./fetch_oclc_dewey_linked_data.pl -v -a SNC -w "authid = 5955" -l it ./fetch_oclc_dewey_linked_data.pl -v -a SNC -w "authid = 5955" -l it -f ./fetch_oclc_dewey_linked_data.pl -v -l no -f ./fetch_oclc_dewey_linked_data.pl ...etc The output of the script for the above commands should resemble what is being depicted in the following asciinema screencast: https://asciinema.org/a/FKxWqNHLeZbxcgCBwYWZ4fod9 If all goes well, you should have updated all authorities that have a DDC field with their matching Dewey Linked Data. |