In MARC 21, having title search field defined in mappings.yaml as 245abp (more than one subfield), the Koha generated title__sort will have not only $a, but also other subfields cut by ind2 characters, i.e.: 245 04 $a Die Renaissance : $b Architektur, Plastik, Malerei, Illustrationen, Zeichnungen becomes as title__sort: Renaissance : itektur, Plastik, Malerei, Illustrationen, Zeichnungen instead of: Renaissance : Architektur, Plastik, Malerei, Illustrationen, Zeichnungen This is because copying an array of pointers in Perl causes you to point to the same (anonymous) element. Therefore, the intended modification only for subfield "a" (Koha/SearchEngine/Elasticsearch.pm, ~line 1157) causes in fact modification for all the subfields sharing the same mapping definition and introduction of the nonfiling_characters_indicator to all subfields 245 when processing 245abp. And this is obviously not what is intended.
Created attachment 148103 [details] [review] Bug 33206 - Bad title__sort made of multifield 245 In MARC 21, having title search field defined in mappings.yaml as 245abp (more than one subfield), the Koha generated title__sort will have not only $a, but also other subfields cut by ind2 characters, i.e.: 245 04 $a Die Renaissance : $b Architektur, Plastik, Malerei, Illustrationen, Zeichnungen becomes as title__sort: Renaissance : itektur, Plastik, Malerei, Illustrationen, Zeichnungen instead of: Renaissance : Architektur, Plastik, Malerei, Illustrationen, Zeichnungen This is because copying an array of pointers in Perl causes you to point to the same (anonymous) element. Therefore, the intended modification only for subfield "a" (Koha/SearchEngine/Elasticsearch.pm, ~line 1157) causes in fact modification for all the subfields sharing the same mapping definition and introduction of the nonfiling_characters_indicator to all subfields 245 when processing 245abp. And this is obviously not what is intended and results (in some cases) with bad sort order when sorting by titles. Test plan ========= 1. Have a standard Koha ES installation. In mappings.yaml you should have 245abp for title search field. 2. Insert / download / modify a record with 245 $b or $p, an article at the beginning of the title, and correctly set 2nd indicator. For instance: 245 04 $a Die Renaissance : $b Architektur, Plastik, Malerei, Illustrationen, Zeichnungen 3. In ES, with Kibana or culr, control the content of title__sort field for the related document. It will be: Renaissance : itektur, Plastik, Malerei, Illustrationen, Zeichnungen 4. Apply the patch, full reindex ES (--rebuild -r -d -a -b) 5. In ES, with Kibana or culr, control the content of title__sort field for the related document. It should be now correct, i.e.: Renaissance : Architektur, Plastik, Malerei, Illustrationen, Zeichnungen
Created attachment 149772 [details] [review] Bug 33206: Bad title__sort made of multifield 245 In MARC 21, having title search field defined in mappings.yaml as 245abp (more than one subfield), the Koha generated title__sort will have not only $a, but also other subfields cut by ind2 characters, i.e.: 245 04 $a Die Renaissance : $b Architektur, Plastik, Malerei, Illustrationen, Zeichnungen becomes as title__sort: Renaissance : itektur, Plastik, Malerei, Illustrationen, Zeichnungen instead of: Renaissance : Architektur, Plastik, Malerei, Illustrationen, Zeichnungen This is because copying an array of pointers in Perl causes you to point to the same (anonymous) element. Therefore, the intended modification only for subfield "a" (Koha/SearchEngine/Elasticsearch.pm, ~line 1157) causes in fact modification for all the subfields sharing the same mapping definition and introduction of the nonfiling_characters_indicator to all subfields 245 when processing 245abp. And this is obviously not what is intended and results (in some cases) with bad sort order when sorting by titles. Test plan ========= 1. Have a standard Koha ES installation. In mappings.yaml you should have 245abp for title search field. 2. Insert / download / modify a record with 245 $b or $p, an article at the beginning of the title, and correctly set 2nd indicator. For instance: 245 04 $a Die Renaissance : $b Architektur, Plastik, Malerei, Illustrationen, Zeichnungen 3. In ES, with Kibana or culr, control the content of title__sort field for the related document. It will be: Renaissance : itektur, Plastik, Malerei, Illustrationen, Zeichnungen 4. Apply the patch, full reindex ES (--rebuild -r -d -a -b) 5. In ES, with Kibana or culr, control the content of title__sort field for the related document. It should be now correct, i.e.: Renaissance : Architektur, Plastik, Malerei, Illustrationen, Zeichnungen Signed-off-by: Nick <nick@bywatersolutions.com>
Created attachment 149773 [details] [review] Bug 33206: Unit test Signed-off-by: Nick <nick@bywatersolutions.com>
Added a unit test, correct commit title, make sure the format is: Bug ####: Description it had a dash not a colon
I just ran into this in training and glad to see there is already a fix underway.
Created attachment 150138 [details] [review] Bug 33206: Bad title__sort made of multifield 245 In MARC 21, having title search field defined in mappings.yaml as 245abp (more than one subfield), the Koha generated title__sort will have not only $a, but also other subfields cut by ind2 characters, i.e.: 245 04 $a Die Renaissance : $b Architektur, Plastik, Malerei, Illustrationen, Zeichnungen becomes as title__sort: Renaissance : itektur, Plastik, Malerei, Illustrationen, Zeichnungen instead of: Renaissance : Architektur, Plastik, Malerei, Illustrationen, Zeichnungen This is because copying an array of pointers in Perl causes you to point to the same (anonymous) element. Therefore, the intended modification only for subfield "a" (Koha/SearchEngine/Elasticsearch.pm, ~line 1157) causes in fact modification for all the subfields sharing the same mapping definition and introduction of the nonfiling_characters_indicator to all subfields 245 when processing 245abp. And this is obviously not what is intended and results (in some cases) with bad sort order when sorting by titles. Test plan ========= 1. Have a standard Koha ES installation. In mappings.yaml you should have 245abp for title search field. 2. Insert / download / modify a record with 245 $b or $p, an article at the beginning of the title, and correctly set 2nd indicator. For instance: 245 04 $a Die Renaissance : $b Architektur, Plastik, Malerei, Illustrationen, Zeichnungen 3. In ES, with Kibana or culr, control the content of title__sort field for the related document. It will be: Renaissance : itektur, Plastik, Malerei, Illustrationen, Zeichnungen 4. Apply the patch, full reindex ES (--rebuild -r -d -a -b) 5. In ES, with Kibana or culr, control the content of title__sort field for the related document. It should be now correct, i.e.: Renaissance : Architektur, Plastik, Malerei, Illustrationen, Zeichnungen Signed-off-by: Nick <nick@bywatersolutions.com> Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Created attachment 150139 [details] [review] Bug 33206: Unit test Signed-off-by: Nick <nick@bywatersolutions.com> Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Nick <nick@bywatersolutions.com> ? Found some other 'oddness' with the title sorting, filed as bug 33594.
Pushed to master for 23.05. Nice work everyone, thanks!
Nice work everyone! Pushed to stable for 22.11.x
Backported to 22.05.x for upcoming 22.05.14
Not backporting for 21.11.x