The duplicate check in cataloguing doesn't work with Elasticsearch. To test: * With Zebra: Download a record via Z39.50 and save. * Make sure the record is found in the catalog. * Download the same record via Z39.50 again and try to save - duplicate confirm message pops up. * Switch to Zebra * Again, make sure that the record can be found in the catalog. * Download the record again via Z39.50 and try to save - no duplicate warning, the record is saved. If an example is needed we could provide a taget and sample. Note: As I understand FindDuplicate it checks for ISSN and after that a combination of title and author. Is there a thing we might be missing here? Tested in different installations with master and 21.11.
Ok, I have found the issue: The problem appears for records with multiple ISBN. An example: * Make sure you are using Elasticsearch * Z39.50 target: LOC * Search for: Control number: 14455023 * Import and save * Search for the record again * Import and save - duplicate check hasn't been triggered Taking a closer look: The duplicate check using FindDuplicate will create the search: isbn:1600213510 | 9781600213519 If you search this in the catalog search, it will yield no results with Elasticsearch, but it will indeed work with Zebra, I guess because the | is treated differently in this case.
I am not sure how the search for Elasticsearch should be built in order to work reliably, help and hints appreciated!
Created attachment 147333 [details] [review] Bug 31471: Send multiple ISBN with 'OR' for FindDuplicate When a record has multiple ISBNs the database stores them seperated by a pipe Zebra interprets a string like "isbn:1600213510 | 9781600213519" as an 'OR' search, but for Elasticsearch we need to explicitly send "OR" - and Zebra support this as well To test: 1 - Make sure you are using Elasticsearch 2 - Cataloging->Add a new record from Z3950 3 - Choose target: LOC 4 - Search for: Control number: 14455023 5 - Import and save 6 - Search for the record again 7 - Import and save - duplicate check isn't been triggered 8 - Apply patch 9 - restart_all 10 - repeat 2-7 11 - this time you should get a duplicate notification 12 - Sitch SearchEngine syspref to 'Zebra' 13 - repeat 2-7 14 - Ensure you still get duplicate notification
Created attachment 147530 [details] [review] Bug 31471: Send multiple ISBN with 'OR' for FindDuplicate When a record has multiple ISBNs the database stores them seperated by a pipe Zebra interprets a string like "isbn:1600213510 | 9781600213519" as an 'OR' search, but for Elasticsearch we need to explicitly send "OR" - and Zebra support this as well To test: 1 - Make sure you are using Elasticsearch 2 - Cataloging->Add a new record from Z3950 3 - Choose target: LOC 4 - Search for: Control number: 14455023 5 - Import and save 6 - Search for the record again 7 - Import and save - duplicate check isn't been triggered 8 - Apply patch 9 - restart_all 10 - repeat 2-7 11 - this time you should get a duplicate notification 12 - Sitch SearchEngine syspref to 'Zebra' 13 - repeat 2-7 14 - Ensure you still get duplicate notification Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Tested with LoC Control No. 1.) 16509983 (100 countries, 5000 ideas), ISBN: 9781426207587 (softcover : alk. paper) :; 1426207581 (softcover : alk. paper) and 2.) 19483458 (Cyberdoc), ISBN: 9780985248314 (cloth); 9780985248321 (pbk.) Thanks for the patch!
The solution can be so easy, once you see it :) Thank you, Nick!
Created attachment 147752 [details] [review] Bug 31471: Send multiple ISBN with 'OR' for FindDuplicate When a record has multiple ISBNs the database stores them seperated by a pipe Zebra interprets a string like "isbn:1600213510 | 9781600213519" as an 'OR' search, but for Elasticsearch we need to explicitly send "OR" - and Zebra support this as well To test: 1 - Make sure you are using Elasticsearch 2 - Cataloging->Add a new record from Z3950 3 - Choose target: LOC 4 - Search for: Control number: 14455023 5 - Import and save 6 - Search for the record again 7 - Import and save - duplicate check isn't been triggered 8 - Apply patch 9 - restart_all 10 - repeat 2-7 11 - this time you should get a duplicate notification 12 - Sitch SearchEngine syspref to 'Zebra' 13 - repeat 2-7 14 - Ensure you still get duplicate notification Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu> Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Pushed to master for 23.05. Nice work everyone, thanks!
Nice work, thanks everyone! Pushed to 22.11.x for the next release.
Backported to 22.05.x for upcoming 22.05.11
applied to 21.11 for 21.11.19
Not backported to 21.05.x