Our library includes the copyright symbol when cataloguing the publication date (264$c) according to RDA 2.11.1.3. Searching dates as free text (not using the advanced date search) is an issue for us in both the opac and staff interface. Koha is not pulling all possible results because we are using the copyright symbol in the record. I've been advised that Koha treats ©2023 and 2023 as two different words. Of course when adding the copyright symbol to the date to the general search, the correct number of results are shown. We don't want to require a user to know alt codes to produce a copyright symbol, and many students simply are not going to use an advanced date search. Is it possible for KOHA to have the ability to ignore a copyright symbol in 264c for free text searching and return search results based on the date alone?
For searching you should catalog the normalized date in 008 7-10, this will be used for date range search and might improve things a bit. I will also attach patches that work for ignoring © with Zebra. I am not sure if a change is needed for Elasticsearch (cannot test it currently) and I would not know where to make the changes. If you are using Elastisearch and can confirm this is a problem there too, would you mind filing another bug for the "Searching - Elasticsearch" component?
Created attachment 149728 [details] [review] Bug 32937: Make Zebra ignore the copyright symbol in searches It's common to catalog the year with the copyright symbol attached to it. This creates some issues with search that we can fix by adding the © to the list of characters Zebra ignores in search. To test: * Search for any existing publication year with ©, example: ©1951 * Verify your record is not found * Add the copyright symbol to a record, verify it's now found with ©, but not without * Apply patch * On ktd: * sudo cp -i /kohadevbox/koha/etc/zebradb/etc/word-phrase-utf.chr /etc/koha/zebradb/etc/word-phrase-utf.chr * sudo koha-zebra --restart kohadev * sudo koha-mysql kohadev * DELETE FROM biblio WHERE biblionumber = 369; * The reindex woudl fail for me with this broken record present. * sudo koha-rebuild-zebra -f kohadev * Repeat searches, the records should be found when searching with and without ©.
Created attachment 149736 [details] [review] Bug 32937: Make Zebra ignore the copyright symbol in searches It's common to catalog the year with the copyright symbol attached to it. This creates some issues with search that we can fix by adding the © to the list of characters Zebra ignores in search. To test: * Search for any existing publication year with ©, example: ©1951 * Verify your record is not found * Add the copyright symbol to a record, verify it's now found with ©, but not without * Apply patch * On ktd: * sudo cp -i /kohadevbox/koha/etc/zebradb/etc/word-phrase-utf.chr /etc/koha/zebradb/etc/word-phrase-utf.chr * sudo koha-zebra --restart kohadev * sudo koha-mysql kohadev * DELETE FROM biblio WHERE biblionumber = 369; * The reindex woudl fail for me with this broken record present. * sudo koha-rebuild-zebra -f kohadev * Repeat searches, the records should be found when searching with and without ©. Signed-off-by: David Nind <david@davidnind.com>
(In reply to Katrin Fischer from comment #1) > I will also attach patches that work for ignoring © with Zebra. I am not > sure if a change is needed for Elasticsearch (cannot test it currently) and > I would not know where to make the changes. If you are using Elastisearch > and can confirm this is a problem there too, would you mind filing another > bug for the "Searching - Elasticsearch" component? I tested with KTD and Elasticsearch and OpenSearch (ktd --es7 up and ktd --os1 up). The record was found with and without the copyright symbol, so I don't think it affects Koha with Elasticsearch or OpenSearch. Note: I couldn't get KTD to start properly using --es6, --es8, and --os2. Testing notes: 1. Edited the BKS framework so that 264$c is visible in the editor. 2. Add a year to 264$c with and without the copyright symbol, as per the test plan.
Created attachment 150404 [details] [review] Bug 32937: Make Zebra ignore the copyright symbol in searches It's common to catalog the year with the copyright symbol attached to it. This creates some issues with search that we can fix by adding the © to the list of characters Zebra ignores in search. To test: * Search for any existing publication year with ©, example: ©1951 * Verify your record is not found * Add the copyright symbol to a record, verify it's now found with ©, but not without * Apply patch * On ktd: * sudo cp -i /kohadevbox/koha/etc/zebradb/etc/word-phrase-utf.chr /etc/koha/zebradb/etc/word-phrase-utf.chr * sudo koha-zebra --restart kohadev * sudo koha-mysql kohadev * DELETE FROM biblio WHERE biblionumber = 369; * The reindex woudl fail for me with this broken record present. * sudo koha-rebuild-zebra -f kohadev * Repeat searches, the records should be found when searching with and without ©. Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Pushed to master for 23.05. Nice work everyone, thanks!
Nice work everyone! Pushed to stable for 22.11.x
Backported to 22.05.x for upcoming 22.05.14
Not backporting to 21.11.x