Currently, there is no integrated way of knowing if an item being requested is already available. If a user creating a request wants to know if the item they are requesting is already available, it is necessary to search the, possibly, multiple sources the library has available (catalogue, discovery interface, open access provider etc.) first. The problem is further compounded if the institution has the unmediated ILL workflow enabled where a request can be placed with no oversight, even though the institution may already have access to it. I propose a mechanism by which, during request creation, one or more "sources" are searched with the provided metadata to try and determine if the item being requested is already available and, if so, links to that item are provided. This behaviour will be optional, via a syspref. "Sources" will be provided as Koha plugins and will provide the ability to accept metadata conforming to the established ILL schema, search the source with which they're concerned, and return results in a prescribed format. This will be performed via an API common to all "source" plugins. All plugins performing this task will be in a common plugin category and therefore enumerable. The workflow for two backends: Freeform: - User fills in Freeform form, clicks submit button - User is taken to an "Availability" page - ILL enumerates source plugins that can service this type of request (mostly determined by material type) and makes search requests to each using provided metadata - Displays results from each source, with links to matching items - User can then either choose a result and be taken to it, or choose none and continue with the request as normal BLDSS: - Identical to above except the user will choose a BLDSS result rather than filling in the Freeform form As part of this work, a source plugin, which will serve as a reference, will be provided.
Just to clarify, in the above workflows, at the stage: - User can then either choose a result and be taken to it, or choose none and continue with the request as normal If no sources return any results, then the user would continue with the request as normal
Created attachment 91464 [details] [review] Bug 23173: Add ILLCheckAvailability syspref This patch adds the new syspref "ILLCheckAvailability"
Created attachment 91465 [details] [review] Bug 23173: Provide core infrastructure This patch adds the required infrastructure to enable ILL availability plugins to intercept the request creation process and, using the supplied metadata, search for and display possible relevant items from whichever availability plugins are installed. Currently two availability plugins exist: z39.50 - Searches any number of the Koha instance's configured Z targets https://github.com/PTFS-Europe/koha-plugin-ill-avail-z3950 Unpaywall - Searches the Unpaywall API for possible open access versions of the requested item https://github.com/PTFS-Europe/koha-plugin-ill-avail-unpaywall The Unpaywall plugin is intended to serve as a "reference" plugin as the API it deals with is extremely simple
Test plan: - Apply bug - Enable the "ILLCheckAvailability" syspref - Install the Unpaywall plugin and configure using your email addresss - Ensure you have a work ILL setup with the FreeForm backend - Create an article FreeForm request containing a valid DOI (ideally one that you know Unpaywall is aware of) - Click "Submit" on the FreeForm request form => TEST: Observe that ILL now displays the availability of the requested item at Unpaywall => TEST: If an item is displayed, observe that clicking the item link takes you directly to the item - Click the "continue creating your request" button => TEST: Observe that the request continues as normal
Created attachment 92107 [details] [review] Bug 23173: Add ILLCheckAvailability syspref This patch adds the new syspref "ILLCheckAvailability"
Created attachment 92108 [details] [review] Bug 23173: Provide core infrastructure This patch adds the required infrastructure to enable ILL availability plugins to intercept the request creation process and, using the supplied metadata, search for and display possible relevant items from whichever availability plugins are installed. Currently two availability plugins exist: z39.50 - Searches any number of the Koha instance's configured Z targets https://github.com/PTFS-Europe/koha-plugin-ill-avail-z3950 Unpaywall - Searches the Unpaywall API for possible open access versions of the requested item https://github.com/PTFS-Europe/koha-plugin-ill-avail-unpaywall The Unpaywall plugin is intended to serve as a "reference" plugin as the API it deals with is extremely simple
Rebased on top of latest master and merge conflicts resolved
Created attachment 92539 [details] [review] Bug 23173: Add ILLCheckAvailability syspref This patch adds the new syspref "ILLCheckAvailability"
Created attachment 92540 [details] [review] Bug 23173: Provide core infrastructure This patch adds the required infrastructure to enable ILL availability plugins to intercept the request creation process and, using the supplied metadata, search for and display possible relevant items from whichever availability plugins are installed. Currently two availability plugins exist: z39.50 - Searches any number of the Koha instance's configured Z targets https://github.com/PTFS-Europe/koha-plugin-ill-avail-z3950 Unpaywall - Searches the Unpaywall API for possible open access versions of the requested item https://github.com/PTFS-Europe/koha-plugin-ill-avail-unpaywall The Unpaywall plugin is intended to serve as a "reference" plugin as the API it deals with is extremely simple
Created attachment 92550 [details] [review] Bug 23173: Add ILLCheckAvailability syspref This patch adds the new syspref "ILLCheckAvailability"
Created attachment 92551 [details] [review] Bug 23173: Provide core infrastructure This patch adds the required infrastructure to enable ILL availability plugins to intercept the request creation process and, using the supplied metadata, search for and display possible relevant items from whichever availability plugins are installed. Currently two availability plugins exist: z39.50 - Searches any number of the Koha instance's configured Z targets https://github.com/PTFS-Europe/koha-plugin-ill-avail-z3950 Unpaywall - Searches the Unpaywall API for possible open access versions of the requested item https://github.com/PTFS-Europe/koha-plugin-ill-avail-unpaywall The Unpaywall plugin is intended to serve as a "reference" plugin as the API it deals with is extremely simple
Created attachment 92640 [details] [review] Bug 23173: Add ILLCheckAvailability syspref This patch adds the new syspref "ILLCheckAvailability"
Created attachment 92641 [details] [review] Bug 23173: Provide core infrastructure This patch adds the required infrastructure to enable ILL availability plugins to intercept the request creation process and, using the supplied metadata, search for and display possible relevant items from whichever availability plugins are installed. Currently three availability plugins exist: z39.50 - Searches any number of the Koha instance's configured Z targets https://github.com/PTFS-Europe/koha-plugin-ill-avail-z3950 EDS - Searches the EBSCO Discovery Service https://github.com/PTFS-Europe/koha-plugin-ill-avail-eds Unpaywall - Searches the Unpaywall API for possible open access versions of the requested item https://github.com/PTFS-Europe/koha-plugin-ill-avail-unpaywall The Unpaywall plugin is intended to serve as a "reference" plugin as the API it deals with is extremely simple
Updated test plan: - Apply bug - Rebuild CSS - Enable the "ILLCheckAvailability" syspref - Ensure you have a working ILL setup with the FreeForm backend - Ensure you have at least one ILL request partner set up. By default, this is patrons in the ILLLIBS category, who have a primary email defined. - Install the Unpaywall availability plugin (https://github.com/PTFS-Europe/koha-plugin-ill-avail-unpaywall) and configure using your email addresss - Install the z39.50 availability plugin (https://github.com/PTFS-Europe/koha-plugin-ill-avail-z3950) and configure at least one target. Ensure you specify the ID of at least one of your request partners, this creates a mapping between that target and the request partner == Main availability searching - Create an article FreeForm request containing a valid DOI (ideally one that you know Unpaywall is aware of) - Click "Submit" on the FreeForm request form => TEST: Observe that ILL now displays the availability of the requested item at Unpaywall => TEST: If an item is displayed, observe that clicking the item link takes you directly to the item - Click the "continue creating your request" button => TEST: Observe that the request continues as normal == Request from partners availability searching - On a request's "Manage request" screen, select "Place request with partners" - On the "Place request with partners" screen, select at least one of the partners with which you associated a Z target in the plugin config => TEST: Observe that when a partner is selected, the "Search selected partners" button appears - Click "Search selected partners" => TEST: Observe that the "Search partners" modal appears and upon the search completing, displays matching results for each all partners who are attached to a Z target.
Created attachment 92693 [details] [review] Bug 23173: Add ILLCheckAvailability syspref This patch adds the new syspref "ILLCheckAvailability"
Created attachment 92694 [details] [review] Bug 23173: Provide core infrastructure This patch adds the required infrastructure to enable ILL availability plugins to intercept the request creation process and, using the supplied metadata, search for and display possible relevant items from whichever availability plugins are installed. Currently three availability plugins exist: z39.50 - Searches any number of the Koha instance's configured Z targets https://github.com/PTFS-Europe/koha-plugin-ill-avail-z3950 EDS - Searches the EBSCO Discovery Service https://github.com/PTFS-Europe/koha-plugin-ill-avail-eds Unpaywall - Searches the Unpaywall API for possible open access versions of the requested item https://github.com/PTFS-Europe/koha-plugin-ill-avail-unpaywall The Unpaywall plugin is intended to serve as a "reference" plugin as the API it deals with is extremely simple
Created attachment 92695 [details] [review] Bug 23173: Add unit tests Test Koha::Illrequests::Availability
Created attachment 92715 [details] [review] Bug 23173: Add ILLCheckAvailability syspref This patch adds the new syspref "ILLCheckAvailability"
Created attachment 92716 [details] [review] Bug 23173: Provide core infrastructure This patch adds the required infrastructure to enable ILL availability plugins to intercept the request creation process and, using the supplied metadata, search for and display possible relevant items from whichever availability plugins are installed. Currently three availability plugins exist: z39.50 - Searches any number of the Koha instance's configured Z targets https://github.com/PTFS-Europe/koha-plugin-ill-avail-z3950 EDS - Searches the EBSCO Discovery Service https://github.com/PTFS-Europe/koha-plugin-ill-avail-eds Unpaywall - Searches the Unpaywall API for possible open access versions of the requested item https://github.com/PTFS-Europe/koha-plugin-ill-avail-unpaywall The Unpaywall plugin is intended to serve as a "reference" plugin as the API it deals with is extremely simple
Created attachment 92717 [details] [review] Bug 23173: Add unit tests Test Koha::Illrequests::Availability
Created attachment 93146 [details] [review] Bug 23173: Add ILLCheckAvailability syspref This patch adds the new syspref "ILLCheckAvailability"
Created attachment 93147 [details] [review] Bug 23173: Provide core infrastructure This patch adds the required infrastructure to enable ILL availability plugins to intercept the request creation process and, using the supplied metadata, search for and display possible relevant items from whichever availability plugins are installed. Currently three availability plugins exist: z39.50 - Searches any number of the Koha instance's configured Z targets https://github.com/PTFS-Europe/koha-plugin-ill-avail-z3950 EDS - Searches the EBSCO Discovery Service https://github.com/PTFS-Europe/koha-plugin-ill-avail-eds Unpaywall - Searches the Unpaywall API for possible open access versions of the requested item https://github.com/PTFS-Europe/koha-plugin-ill-avail-unpaywall The Unpaywall plugin is intended to serve as a "reference" plugin as the API it deals with is extremely simple
Created attachment 93148 [details] [review] Bug 23173: Add unit tests Test Koha::Illrequests::Availability
Created attachment 93909 [details] [review] Bug 23173: Add ILLCheckAvailability syspref This patch adds the new syspref "ILLCheckAvailability" Sponsored-by: Public Health England
Created attachment 93910 [details] [review] Bug 23173: Provide core infrastructure This patch adds the required infrastructure to enable ILL availability plugins to intercept the request creation process and, using the supplied metadata, search for and display possible relevant items from whichever availability plugins are installed. Currently three availability plugins exist: z39.50 - Searches any number of the Koha instance's configured Z targets https://github.com/PTFS-Europe/koha-plugin-ill-avail-z3950 EDS - Searches the EBSCO Discovery Service https://github.com/PTFS-Europe/koha-plugin-ill-avail-eds Unpaywall - Searches the Unpaywall API for possible open access versions of the requested item https://github.com/PTFS-Europe/koha-plugin-ill-avail-unpaywall The Unpaywall plugin is intended to serve as a "reference" plugin as the API it deals with is extremely simple
Created attachment 93911 [details] [review] Bug 23173: Add unit tests Test Koha::Illrequests::Availability
Created attachment 95268 [details] [review] Bug 23173: Add ILLCheckAvailability syspref This patch adds the new syspref "ILLCheckAvailability" Sponsored-by: Public Health England
Created attachment 95269 [details] [review] Bug 23173: Provide core infrastructure This patch adds the required infrastructure to enable ILL availability plugins to intercept the request creation process and, using the supplied metadata, search for and display possible relevant items from whichever availability plugins are installed. Currently three availability plugins exist: z39.50 - Searches any number of the Koha instance's configured Z targets https://github.com/PTFS-Europe/koha-plugin-ill-avail-z3950 EDS - Searches the EBSCO Discovery Service https://github.com/PTFS-Europe/koha-plugin-ill-avail-eds Unpaywall - Searches the Unpaywall API for possible open access versions of the requested item https://github.com/PTFS-Europe/koha-plugin-ill-avail-unpaywall The Unpaywall plugin is intended to serve as a "reference" plugin as the API it deals with is extremely simple
Created attachment 95270 [details] [review] Bug 23173: Add unit tests Test Koha::Illrequests::Availability
Needed a rebase
The test plans says "Create an article FreeForm request containing a valid DOI (ideally one that you know Unpaywall is aware of)", it turns out finding a DOI that Unpaywall is aware of is harder than you think, so: 10.1038/nature12373
Created attachment 98521 [details] [review] Bug 23173: (follow-up) Allow for wide characters The conversion of the metadata into Base64 for passing in the API call to the plugin API routes was failing if the metadata contained a wide character, so we now encode it into UTF-8 first
Created attachment 99083 [details] [review] Bug 23173: (follow-up) Allow for customisable display name We now allow for the service to return a name that can be specified in the plugin config. This patch switches to using that, rather than just the plugin name defined in the plugin's metadata hashref
Created attachment 100724 [details] [review] Bug 23173: (follow-up) Allow for customisable display name We now allow for the service to return a name that can be specified in the plugin config. This patch switches to using that, rather than just the plugin name defined in the plugin's metadata hashref
Created attachment 100725 [details] [review] Bug 23173: (follow-up) Allow for customisable display name We now allow for the service to return a name that can be specified in the plugin config. This patch switches to using that, rather than just the plugin name defined in the plugin's metadata hashref
Created attachment 100740 [details] [review] Bug 23173: (follow-up) Display available partners When selecting partners for search during the "Place request with partner libraries" workflow, there was no indication of which partners could actually be searched (i.e. are attached to a Z target and enabled for partner search). We now list them
Created attachment 102179 [details] [review] Bug 23173: Add ILLCheckAvailability syspref This patch adds the new syspref "ILLCheckAvailability" Sponsored-by: Public Health England Signed-off-by: Niamh Walker-Headon <Niamh.Walker-Headon@tudublin.ie>
Created attachment 102180 [details] [review] Bug 23173: Provide core infrastructure This patch adds the required infrastructure to enable ILL availability plugins to intercept the request creation process and, using the supplied metadata, search for and display possible relevant items from whichever availability plugins are installed. Currently three availability plugins exist: z39.50 - Searches any number of the Koha instance's configured Z targets https://github.com/PTFS-Europe/koha-plugin-ill-avail-z3950 EDS - Searches the EBSCO Discovery Service https://github.com/PTFS-Europe/koha-plugin-ill-avail-eds Unpaywall - Searches the Unpaywall API for possible open access versions of the requested item https://github.com/PTFS-Europe/koha-plugin-ill-avail-unpaywall The Unpaywall plugin is intended to serve as a "reference" plugin as the API it deals with is extremely simple Signed-off-by: Niamh Walker-Headon <Niamh.Walker-Headon@tudublin.ie>
Created attachment 102181 [details] [review] Bug 23173: Add unit tests Test Koha::Illrequests::Availability Signed-off-by: Niamh Walker-Headon <Niamh.Walker-Headon@tudublin.ie>
Created attachment 102182 [details] [review] Bug 23173: (follow-up) Allow for wide characters The conversion of the metadata into Base64 for passing in the API call to the plugin API routes was failing if the metadata contained a wide character, so we now encode it into UTF-8 first Signed-off-by: Niamh Walker-Headon <Niamh.Walker-Headon@tudublin.ie>
Created attachment 102183 [details] [review] Bug 23173: (follow-up) Allow for customisable display name We now allow for the service to return a name that can be specified in the plugin config. This patch switches to using that, rather than just the plugin name defined in the plugin's metadata hashref Signed-off-by: Niamh Walker-Headon <Niamh.Walker-Headon@tudublin.ie>
Created attachment 102184 [details] [review] Bug 23173: (follow-up) Display available partners When selecting partners for search during the "Place request with partner libraries" workflow, there was no indication of which partners could actually be searched (i.e. are attached to a Z target and enabled for partner search). We now list them Signed-off-by: Niamh Walker-Headon <Niamh.Walker-Headon@tudublin.ie>
Created attachment 102286 [details] [review] Bug 23173: Add ILLCheckAvailability syspref This patch adds the new syspref "ILLCheckAvailability" Sponsored-by: Public Health England Signed-off-by: Niamh Walker-Headon <Niamh.Walker-Headon@tudublin.ie> Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Created attachment 102287 [details] [review] Bug 23173: Provide core infrastructure This patch adds the required infrastructure to enable ILL availability plugins to intercept the request creation process and, using the supplied metadata, search for and display possible relevant items from whichever availability plugins are installed. Currently three availability plugins exist: z39.50 - Searches any number of the Koha instance's configured Z targets https://github.com/PTFS-Europe/koha-plugin-ill-avail-z3950 EDS - Searches the EBSCO Discovery Service https://github.com/PTFS-Europe/koha-plugin-ill-avail-eds Unpaywall - Searches the Unpaywall API for possible open access versions of the requested item https://github.com/PTFS-Europe/koha-plugin-ill-avail-unpaywall The Unpaywall plugin is intended to serve as a "reference" plugin as the API it deals with is extremely simple Signed-off-by: Niamh Walker-Headon <Niamh.Walker-Headon@tudublin.ie> Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Created attachment 102288 [details] [review] Bug 23173: Add unit tests Test Koha::Illrequests::Availability Signed-off-by: Niamh Walker-Headon <Niamh.Walker-Headon@tudublin.ie> Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Created attachment 102289 [details] [review] Bug 23173: (follow-up) Allow for wide characters The conversion of the metadata into Base64 for passing in the API call to the plugin API routes was failing if the metadata contained a wide character, so we now encode it into UTF-8 first Signed-off-by: Niamh Walker-Headon <Niamh.Walker-Headon@tudublin.ie> Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Created attachment 102290 [details] [review] Bug 23173: (follow-up) Allow for customisable display name We now allow for the service to return a name that can be specified in the plugin config. This patch switches to using that, rather than just the plugin name defined in the plugin's metadata hashref Signed-off-by: Niamh Walker-Headon <Niamh.Walker-Headon@tudublin.ie> Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Created attachment 102291 [details] [review] Bug 23173: (follow-up) Display available partners When selecting partners for search during the "Place request with partner libraries" workflow, there was no indication of which partners could actually be searched (i.e. are attached to a Z target and enabled for partner search). We now list them Signed-off-by: Niamh Walker-Headon <Niamh.Walker-Headon@tudublin.ie> Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Created attachment 102292 [details] [review] Bug 23173: (follow-up) Small improvements in efficiency Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Created attachment 102293 [details] [review] Bug 23173: (follow-up) Fix QA script errors Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Created attachment 102294 [details] [review] Bug 23173: (follow-up) Fix QA script errors Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Nice work everyone! Pushed to master for 20.05
enhancement not backported to 19.11.x