Bug 36554

Summary: Document languages from field 041 should be present in 'ln' search field and Languages facet (MARC 21)
Product: Koha Reporter: Janusz Kaczmarek <januszop>
Component: Searching - ElasticsearchAssignee: Janusz Kaczmarek <januszop>
Status: Pushed to oldstable --- QA Contact:
Severity: minor    
Priority: P5 - low CC: fridolin.somers, lucas, martin.renvoize, mspinney, nick
Version: Main   
Hardware: All   
OS: All   
Change sponsored?: --- Patch complexity: Trivial patch
Documentation contact: Documentation submission:
Text to go in the release notes:
Version(s) released in:
24.05.00,23.11.06,23.05.12
Attachments: Bug 36554: Document languages from field 041 should be present in 'ln' search field and Languages facet (MARC 21)
Bug 36554: Document languages from field 041 should be present in 'ln' search field and Languages facet (MARC 21)

Description Janusz Kaczmarek 2024-04-09 14:18:54 UTC
In Elasticsearch, the field 041 (subfields: a, d, e, i, j) is not indexed with 'ln' search field. As a result records cannot be found when searching with languages present in 041 (but only with the one from 008/35-37), and the languages are also missing from the facet.

Subfields content (only relevant subfields):
$a - Language code of text/sound track or separate title
$d - Language code of sung or spoken text
$e - Language code of librettos
$i - Language code of intertitles
$j - Language code of subtitles

All these subfields are repeatable. 

The most important is of course 041 $a but the rest could be also of interest.
Comment 1 Janusz Kaczmarek 2024-04-11 08:49:22 UTC
Created attachment 164664 [details] [review]
Bug 36554: Document languages from field 041 should be present in 'ln' search field and Languages facet (MARC 21)

In Elasticsearch, the field 041 (subfields: a, d, e, i, j)
is not indexed with 'ln' search field.  As a result, records cannot
be found when searching with languages present in 041 (but only
with the one from 008/35-37), and the languages are also missing
from the facet.

Subfields content (only relevant subfields):
$a - Language code of text/sound track or separate title
$d - Language code of sung or spoken text
$e - Language code of librettos
$i - Language code of intertitles
$j - Language code of subtitles

Test plan
=========
0. Have a test installation with Elasticsearch.
1. In ktd with its test data, make a biblio search for a language present
   in 041 a/d/e/i/j but not in 008/35-37, e.g. for Japanese (with ln:jpn
   or from Advance search).  You will get no results.
2. Apply the patch, reindex with:
   sudo koha-elasticsearch --rebuild -r -b kohadev
3. Repeat the test. You should get some records and also you should see
   the Japanese language in the Languages facet.
Comment 2 Roman Dolny 2024-04-16 16:40:26 UTC
Created attachment 164959 [details] [review]
Bug 36554: Document languages from field 041 should be present in 'ln' search field and Languages facet (MARC 21)

In Elasticsearch, the field 041 (subfields: a, d, e, i, j)
is not indexed with 'ln' search field.  As a result, records cannot
be found when searching with languages present in 041 (but only
with the one from 008/35-37), and the languages are also missing
from the facet.

Subfields content (only relevant subfields):
$a - Language code of text/sound track or separate title
$d - Language code of sung or spoken text
$e - Language code of librettos
$i - Language code of intertitles
$j - Language code of subtitles

Test plan
=========
0. Have a test installation with Elasticsearch.
1. In ktd with its test data, make a biblio search for a language present
   in 041 a/d/e/i/j but not in 008/35-37, e.g. for Japanese (with ln:jpn
   or from Advance search).  You will get no results.
2. Apply the patch, reindex with:
   sudo koha-elasticsearch --rebuild -r -b kohadev
3. Repeat the test. You should get some records and also you should see
   the Japanese language in the Languages facet.

Signed-off-by: Roman Dolny <roman.dolny@jezuici.pl>
Comment 3 Katrin Fischer 2024-05-06 09:13:45 UTC
Picked out of QA queue.
Comment 4 Katrin Fischer 2024-05-06 09:19:05 UTC
Pushed for 24.05!

Well done everyone, thank you!
Comment 5 Fridolin Somers 2024-05-24 14:05:02 UTC
Pushed to 23.11.x for 23.11.06
Comment 6 Lucas Gass 2024-05-28 21:43:20 UTC
Backported to 23.05.x for upcoming 23.05.12