Bug 22339 - Elasticsearch - fixed field mappings should match MARC ranges
Summary: Elasticsearch - fixed field mappings should match MARC ranges
Status: CLOSED FIXED
Alias: None
Product: Koha
Classification: Unclassified
Component: Searching - Elasticsearch (show other bugs)
Version: Main
Hardware: All All
: P5 - low normal (vote)
Assignee: Ere Maijala
QA Contact:
URL:
Keywords:
: 22256 (view as bug list)
Depends on:
Blocks:
 
Reported: 2019-02-14 13:27 UTC by Nick Clemens
Modified: 2020-06-04 20:32 UTC (History)
6 users (show)

See Also:
Change sponsored?: ---
Patch complexity: Small patch
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:
19.05.00, 18.11.05


Attachments
Bug 22339: Fix search field mappings of MARC fixed fields (9.20 KB, patch)
2019-03-14 11:37 UTC, Ere Maijala
Details | Diff | Splinter Review
Bug 22339: Fix search field mappings of MARC fixed fields (9.32 KB, patch)
2019-04-12 13:00 UTC, Björn Nylén
Details | Diff | Splinter Review
Bug 22339: Fix search field mappings of MARC fixed fields (9.39 KB, patch)
2019-04-25 08:02 UTC, Martin Renvoize
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Nick Clemens 2019-02-14 13:27:27 UTC
ff7-00 is set to: 007_/1
it should be mapped to position 0 I believe
any other specific character positions should be checked to ensure they are corect as well

To recreate:
1 - Find an item with an 007 in your catalog
2 - Make sure it is indexed in ES
3 - View the ES record:
  kohadevbox
    curl 'localhost:9200/koha_kohadev_biblios/data/19?pretty=yes' | grep -A7 ff7
  koha testing docker
    curl 'es:9200/koha_kohadev_biblios/data/19?pretty=yes' | grep -A7 ff7
  adjust for other setups
4 - Note the 00 indexes the 01 position, etc
Comment 1 Ere Maijala 2019-03-14 11:24:11 UTC
*** Bug 22256 has been marked as a duplicate of this bug. ***
Comment 2 Ere Maijala 2019-03-14 11:37:28 UTC
Created attachment 86597 [details] [review]
Bug 22339: Fix search field mappings of MARC fixed fields

Fixes handling of /0 selector in addition to several fixed field positions. Note that ff7-01-02 is in fact 00-01 in Zebra and that has been replicated here.

Test plan:
1. Before applying a patch, check from Elasticsearch (e.g. localhost:9200/koha_biblios/_search?q=_id:123) what is indexed in Elasticsearch for a record for date-entered-on-file, ff7-00, ff7-01, ff7-02 and lleader.
2. Apply the patch, update database and save the record again.
3. Verify that the contents of the forementioned fields is now correct in Elasticsearch.
4. Verify that tests pass:  prove -v t/Koha/SearchEngine/Elasticsearch.t
Comment 3 Ere Maijala 2019-03-14 11:38:36 UTC
Note that it seems ff7-01-02 was misnamed in Zebra, and that is reflected here. It contains positions 00-01.
Comment 4 Frank Hansen 2019-03-29 13:57:53 UTC
Note that leader-fields should be spelled "LDR".
Comment 5 Ere Maijala 2019-04-01 07:31:46 UTC
Frank, sorry, I'm not following. Can you explain with a bit more detail?
Comment 6 Frank Hansen 2019-04-03 08:06:55 UTC
It is a little bit confusing because according to the MARCXML standard, leader field is called "leader". But I discovered that in ES, leader fields obviously is marked up as "LDR" instead of "leader". I have no idea why. Maybe it's a Koha or a Catmandu thing. The thing is, ES fields that comes from content in the leader will not be indexed unless the mapping is corrected to "LDR".
Comment 7 Björn Nylén 2019-04-12 13:00:24 UTC
Created attachment 87905 [details] [review]
Bug 22339: Fix search field mappings of MARC fixed fields

Fixes handling of /0 selector in addition to several fixed field positions. Note that ff7-01-02 is in fact 00-01 in Zebra and that has been replicated here.

Test plan:
1. Before applying a patch, check from Elasticsearch (e.g. localhost:9200/koha_biblios/_search?q=_id:123) what is indexed in Elasticsearch for a record for date-entered-on-file, ff7-00, ff7-01, ff7-02 and lleader.
2. Apply the patch, update database and save the record again.
3. Verify that the contents of the forementioned fields is now correct in Elasticsearch.
4. Verify that tests pass:  prove -v t/Koha/SearchEngine/Elasticsearch.t

Signed-off-by: Björn Nylén <bjorn.nylen@ub.lu.se>
Comment 8 Frank Hansen 2019-04-12 13:06:23 UTC
Yes, it seems to work after all. 
Leader fields is indexed as it should. Perfekt!
Comment 9 Martin Renvoize 2019-04-25 08:02:39 UTC
Created attachment 88706 [details] [review]
Bug 22339: Fix search field mappings of MARC fixed fields

Fixes handling of /0 selector in addition to several fixed field positions. Note that ff7-01-02 is in fact 00-01 in Zebra and that has been replicated here.

Test plan:
1. Before applying a patch, check from Elasticsearch (e.g. localhost:9200/koha_biblios/_search?q=_id:123) what is indexed in Elasticsearch for a record for date-entered-on-file, ff7-00, ff7-01, ff7-02 and lleader.
2. Apply the patch, update database and save the record again.
3. Verify that the contents of the forementioned fields is now correct in Elasticsearch.
4. Verify that tests pass:  prove -v t/Koha/SearchEngine/Elasticsearch.t

Signed-off-by: Björn Nylén <bjorn.nylen@ub.lu.se>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 10 Martin Renvoize 2019-04-25 08:03:38 UTC
I'll confess to not being a total expert when it comes to MARC fields, but these changes appear to be solid and I can't see any obvious regressions.

I really like that we're indexing by '0' now too.

Passing QA.
Comment 11 Nick Clemens 2019-04-25 11:53:05 UTC
Awesome work all!

Pushed to master for 19.05
Comment 12 Martin Renvoize 2019-04-26 15:23:19 UTC
Pushed to 18.11.x for 18.11.05
Comment 13 Lucas Gass 2019-05-24 15:33:43 UTC
not backporting ES bugs/fixes to 18.05.x series