Summary: | Remove extraneous 246 subfields from the title mappings (Elasticsearch, MARC21) | ||
---|---|---|---|
Product: | Koha | Reporter: | Caroline Cyr La Rose <caroline.cyr-la-rose> |
Component: | Cataloging | Assignee: | Caroline Cyr La Rose <caroline.cyr-la-rose> |
Status: | Pushed to oldoldstable --- | QA Contact: | Nick Clemens (kidclamp) <nick> |
Severity: | normal | ||
Priority: | P5 - low | CC: | fridolin.somers, jonathan.druart, lucas, m.de.rooy, martin.renvoize, phil |
Version: | Main | ||
Hardware: | All | ||
OS: | All | ||
See Also: | https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=15248 | ||
Change sponsored?: | --- | Patch complexity: | --- |
Documentation contact: | Documentation submission: | ||
Text to go in the release notes: |
This patch limits indexing of field 246 to $a, $b, $n, and $p in various title indexes.
Previously, all 246 subfields were indexed, including non-title subfields such as $i (Display text), $g (Miscellaneous information), and linking subfields, making the title index very large and giving false results, especially when looking for duplicates in cataloging.
|
Version(s) released in: |
24.11.00,24.05.06,23.11.11
|
Circulation function: | |||
Attachments: |
Screenshot
Bug 37871: Remove extraneous 246 subfields from the title mappings (Elasticsearch, MARC21) Bug 37871: Remove extraneous 246 subfields from the title mappings (Elasticsearch, MARC21) Bug 37871: Remove extraneous 246 subfields from the title mappings (Elasticsearch, MARC21) |
Description
Caroline Cyr La Rose
2024-09-09 16:55:24 UTC
Created attachment 171219 [details]
Screenshot
Screenshot of my record with the title field showing (Title, Number of part, Name of part, Remainder of title) and the "Duplicate record suspected" warning message at the top saying this is a possible duplicate of Giulio Cesare.
Why? That record has 246 $iOther title on container:$aGiulio Cesare in Egitto, and Koha indexes 246$i (and for that matter $h, or any imaginary subfield you add to 246 in a framework) in the title index. The "fun" thing about ElasticSearch is you can fix that: Administration - Search engine configuration (Elasticsearch) - Bibliographic records tab, find the title line for 246 and change it from just 246 to 246ab and koha-elasticsearch --rebuild -d -b kohadev. Thanks Phil! I feel like 246$i should not be indexed in the title, is it just me? We could change the default mapping to 246abnp. Opinions? (In reply to Caroline Cyr La Rose from comment #3) > Thanks Phil! > > I feel like 246$i should not be indexed in the title, is it just me? We > could change the default mapping to 246abnp. > > Opinions? We have had libraries complaining about the title index being too wide in the past. I am not sure if we can make everyone happy here, but maybe removing the $i by default from 246 would be a good start. Created attachment 171488 [details] [review] Bug 37871: Remove extraneous 246 subfields from the title mappings (Elasticsearch, MARC21) This patch limits indexing of field 246 to $a, $b, $n, and $p in various title indexes. Previously, all 246 subfields were indexed, including non-title subfields such as $i (Display text), $g (Miscellaneous information), and linking subfields, making the title index very large and giving false results. To test: 1. Add all the 246 subfields to the default bibliographic framework 1.1. Go to Administration > MARC bibliographic framework 1.2. Click Actions > MARC structure next to the Default framework 1.3. Search for 246 1.4. Click Actions > Edit subfields 1.5. For each subfield, make sure Editor is checked 1.6. Click Save changes 2. Create a new record and fill out all the 246 subfields 2.1. Go to Cataloging 2.2. Click New record 2.3. Fill out the mandatory fields (000, 003, 005, 008, 040$c, 245$a, 942$c) 2.4. Fill out all the subfields in 246 (I simply write the name of the subfield in the text field e.g. Display text in 246$i) 2.5. Click Save 2.6. Click Normal to access the detailed record 3. View the Elastic search record --> All the subfields (including "Display text", "Miscellaneous information" and other non-title subfields) should be indexed in - title - title__suggestion - title-abbreviated - title-abbreviated__sort - title expanded - title-expanded__sort - title-former - title-former__sort 4. Make a title or keyword search for "Display" (or whatever you wrote in 246$i) --> Your record should be in the results 5. Apply patch 6. Rebuild the index using -r koha-elasticsearch --rebuild -d -r -v kohadev 7. Redo step 3 --> Only 246 $a, $b, $n, and $p should be indexed 8. Redo step 4 --> Your record should NOT be in the results Created attachment 171497 [details] [review] Bug 37871: Remove extraneous 246 subfields from the title mappings (Elasticsearch, MARC21) This patch limits indexing of field 246 to $a, $b, $n, and $p in various title indexes. Previously, all 246 subfields were indexed, including non-title subfields such as $i (Display text), $g (Miscellaneous information), and linking subfields, making the title index very large and giving false results. To test: 1. Add all the 246 subfields to the default bibliographic framework 1.1. Go to Administration > MARC bibliographic framework 1.2. Click Actions > MARC structure next to the Default framework 1.3. Search for 246 1.4. Click Actions > Edit subfields 1.5. For each subfield, make sure Editor is checked 1.6. Click Save changes 2. Create a new record and fill out all the 246 subfields 2.1. Go to Cataloging 2.2. Click New record 2.3. Fill out the mandatory fields (000, 003, 005, 008, 040$c, 245$a, 942$c) 2.4. Fill out all the subfields in 246 (I simply write the name of the subfield in the text field e.g. Display text in 246$i) 2.5. Click Save 2.6. Click Normal to access the detailed record 3. View the Elastic search record --> All the subfields (including "Display text", "Miscellaneous information" and other non-title subfields) should be indexed in - title - title__suggestion - title-abbreviated - title-abbreviated__sort - title expanded - title-expanded__sort - title-former - title-former__sort 4. Make a title or keyword search for "Display" (or whatever you wrote in 246$i) --> Your record should be in the results 5. Apply patch 6. Rebuild the index using -r koha-elasticsearch --rebuild -d -r -v kohadev 7. Redo step 3 --> Only 246 $a, $b, $n, and $p should be indexed 8. Redo step 4 --> Your record should NOT be in the results Signed-off-by: Phil Ringnalda <phil@chetcolibrary.org> Created attachment 172165 [details] [review] Bug 37871: Remove extraneous 246 subfields from the title mappings (Elasticsearch, MARC21) This patch limits indexing of field 246 to $a, $b, $n, and $p in various title indexes. Previously, all 246 subfields were indexed, including non-title subfields such as $i (Display text), $g (Miscellaneous information), and linking subfields, making the title index very large and giving false results. To test: 1. Add all the 246 subfields to the default bibliographic framework 1.1. Go to Administration > MARC bibliographic framework 1.2. Click Actions > MARC structure next to the Default framework 1.3. Search for 246 1.4. Click Actions > Edit subfields 1.5. For each subfield, make sure Editor is checked 1.6. Click Save changes 2. Create a new record and fill out all the 246 subfields 2.1. Go to Cataloging 2.2. Click New record 2.3. Fill out the mandatory fields (000, 003, 005, 008, 040$c, 245$a, 942$c) 2.4. Fill out all the subfields in 246 (I simply write the name of the subfield in the text field e.g. Display text in 246$i) 2.5. Click Save 2.6. Click Normal to access the detailed record 3. View the Elastic search record --> All the subfields (including "Display text", "Miscellaneous information" and other non-title subfields) should be indexed in - title - title__suggestion - title-abbreviated - title-abbreviated__sort - title expanded - title-expanded__sort - title-former - title-former__sort 4. Make a title or keyword search for "Display" (or whatever you wrote in 246$i) --> Your record should be in the results 5. Apply patch 6. Rebuild the index using -r koha-elasticsearch --rebuild -d -r -v kohadev 7. Redo step 3 --> Only 246 $a, $b, $n, and $p should be indexed 8. Redo step 4 --> Your record should NOT be in the results Signed-off-by: Phil Ringnalda <phil@chetcolibrary.org> Signed-off-by: Nick Clemens <nick@bywatersolutions.com> Thanks for all the hard work! Pushed to main for the next 24.11.00 release as RM Assistant I am all OK with the change and the push, but I feel the bug description no longer really matches what we did here and maybe a short explanation in release notes would be good as well. Is the title better, and release notes clear? I like it, thanks :) Backported to 24.05.x for upcoming 24.05.06 Pushed to 23.11.x for 23.11.11 |