We would like to add $v volume of work to the title-series elasticsearch index, or 'Series title' option in the advanced search dropdown. So the mappings would become as follows: * 440av * 490av * 830av
Created attachment 167260 [details] [review] Bug 36850: Add volume of work $v to Elasticsearch series title MARC21 index mappings Add $v volume of work to the title-series elasticsearch index, or 'Series title' option in the advanced search dropdown. So the mappings would become as follows: 440av, 490av, 830av Sponsored-by: Educational Services Australia SCIS
Created attachment 167297 [details] [review] Bug 36850: Add volume of work $v to Elasticsearch series title MARC21 index mappings Add $v volume of work to the title-series elasticsearch index, or 'Series title' option in the advanced search dropdown. So the mappings would become as follows: 440av, 490av, 830av Sponsored-by: Educational Services Australia SCIS Signed-off-by: David Nind <david@davidnind.com>
Testing notes using KTD: 1. Start up KTD with Elasticsearch: ktd --es8 up 2. Edit a record and add unique values to 440av, 490av, 830av: . Note: may need to edit the framework for the record to make these tags and subfields visible in the editor . 440$a 987654321 . 440$v 123456789 . 490$a (already visible in BKS framework) 741852963 . 490$v (already visible in BKS framework) 369258147 . 830$a 784951623 . 830$v 326159487 3. Do a search for the terms in subfields $a and $v using the 'Series title' index in the staff interface advanced search. ==> Result: Should get a result when the $a subfield values are used, but no result when the values in the $v subfield are used. 4. Apply this patch and restart all services (restart_all). 5. Reset the mappings: Administration > Catalog > Search engine configuration (Elasticsearch) > Reset mappings (at the bottom of the page) 6. Reindex: koha-elasticsearch --rebuild -d -b -a kohadev 7. Alternative to steps 5 and 6: reset_all then repeat step 2 8. Repeat step 3 ==> Result: Should now get a result when both the $a and $v subfield values are used.
Created attachment 169064 [details] [review] Bug 36850: Add volume of work $v to Elasticsearch series title MARC21 index mappings Add $v volume of work to the title-series elasticsearch index, or 'Series title' option in the advanced search dropdown. So the mappings would become as follows: 440av, 490av, 830av Sponsored-by: Educational Services Australia SCIS Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Thomas Klausner <domm@plix.at>
Can you speak to this one a bit more? Is it a case where a particular search works with Zebra but not Elasticsearch? -- Looking at ./etc/zebradb/marc_defs/marc21/biblios/biblio-zebra-indexdefs.xsl it looks like 440a and 490a get indexed into Title-series:w and Title-series:p. 440a and 490a are also added to the se:0 facet. But then it also looks like the whole of 440 and 490 gets indexed into Title:w and Title-series:w. With 830 the whole things gets indexed into Title:w, Title-series:w, Title-series:p... -- If we wanted to be the most consistent between Zebra and Elasticsearch we'd probably index the whole fields into title-series, although looking at admin/searchengine/elasticsearch/mappings.yaml I'm not sure that you can define facets separately from indexes. -- I haven't loooked yet, but I'm curious if this patch changes how Series facets appear (in a negative way).
Going to move this one into discussion, as it has some unintended side effects.
Created attachment 169069 [details] $v shows up as separate facet value
Created attachment 169070 [details] title-series indexed differently to how one might expect
I think the idea behind this change makes sense, as it would bring it more into line with Zebra. However, it looks like the design of "admin/searchengine/elasticsearch/mappings.yaml" prevents it from working as hoped. If you look at the screenshots above, you'll see in the example that 490$a and 490$v are indexed *separately* for the "title-series" index and the "title-series__facet" index. This has a few consequences. 1. As noted in the screenshot, it means that there is now a facet for the 490$v value of "v1". If you have a lot of 490$v values, your facets are going to be polluted quite badly. 2. For title-series, you'll get the right results when searching "Addison-Wesley professional computing series ; v1" and "Addison-Wesley professional computing series", but if you search "v1", you'll also get a lot of results. This probably isn't intended, but practically speaking it's not a big issue. In fact, it could potentially be handy. Maybe you forgot the series name, but you know the author and the fact that you want v1. So that's kind of cool actually. -- So yeah... I'd say my issue is the facets like I feared in Comment 5.
And actually looking through the default mappings with facets there is only 1 that has multiple subfields defined: "ln (Languages)" having "041adeij" is OK/good because they're all separate codes (rather than having meaning as one long string). There are some default mappings with facets that try to do whole MARC fields: "subject (Topics)" : 611 "subject (Topics)" : 651 That's probably not a good thing, as a lot of irrelevant subfields will end up in facets for people using 611 and 651...
Anyway... like I was saying... I think this is a Koha design problem rather than a problem with this patch per se. But I think it is a blocker. I'll bring it up with people on Mattermost. I think it's an issue that has just gone unnoticed until now.
I am a bit hesitant about the "In dicussion", but we certainly need to find a way so that the $v is not affecting the facets. Could using a separate index for the facet be an OK way to solve this for now? Configuration of facets with Zebra was quite different, so probably can't copy from there, but need to check how we can achieve what we want with Elastic: - enable people to search for $v when using the series search option - display facets using the series title from $a only, so all volumes can be found
(In reply to Katrin Fischer from comment #12) > I am a bit hesitant about the "In dicussion", but we certainly need to find > a way so that the $v is not affecting the facets. > > Could using a separate index for the facet be an OK way to solve this for > now? I don't know the internals well enough, but in theory I think that should work! If I understand correctly, it might look a bit funny on /cgi-bin/koha/admin/searchengine/elasticsearch/mappings.pl but it's a good idea to try
The sponsoring organisation has decided not to go ahead with this, so we'll move on from the bug report now.