Bug 23671 - Elasticsearch shouldn't throw exception on an uppercase subfield identifier
Summary: Elasticsearch shouldn't throw exception on an uppercase subfield identifier
Status: Pushed to master
Alias: None
Product: Koha
Classification: Unclassified
Component: Searching - Elasticsearch (show other bugs)
Version: master
Hardware: All All
: P5 - low trivial (vote)
Assignee: Nicolas Legrand
QA Contact: Nick Clemens
URL:
Keywords:
Depends on: 23670
Blocks:
  Show dependency treegraph
 
Reported: 2019-09-23 16:53 UTC by Nicolas Legrand
Modified: 2019-10-28 09:08 UTC (History)
4 users (show)

See Also:
Change sponsored?: ---
Patch complexity: ---
Who signed the patch off:
Text to go in the release notes:
Version(s) released in:
19.11.00


Attachments
Bug 23671: Elasticsearch shouldn't throw exception on an uppercase subfield identifier (1.72 KB, patch)
2019-09-23 17:05 UTC, Nicolas Legrand
Details | Diff | Splinter Review
Bug 23671: Elasticsearch shouldn't throw exception on an uppercase subfield identifier (1.78 KB, patch)
2019-10-01 13:12 UTC, Séverine Queune
Details | Diff | Splinter Review
Bug 23671: Elasticsearch shouldn't throw exception on an uppercase subfield identifier (1.84 KB, patch)
2019-10-04 10:01 UTC, Nick Clemens
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Nicolas Legrand 2019-09-23 16:53:32 UTC
MARC 21 specifies a subfield identifier “may be any any lowercase alphabetic or numeric character” (https://www.loc.gov/marc/96principl.html section 8.4.2).

It is not that clear for Unimarc or even for ISO 2709 itself. Our catalogue in Unimarc use uppercase alphabetic subfield identifier for items. When we try to index it with Elasticsearch, it throws an exception (well, it tries to, this is fixed by Bug 23670).

We think Elasticsearch shouldn't be picky about uppercase alphabetic subfield identifier, especially when Koha permits their creation in the MARC Framework or in the Elasticsearch mapping.
Comment 1 Nicolas Legrand 2019-09-23 17:05:04 UTC
Created attachment 93133 [details] [review]
Bug 23671: Elasticsearch shouldn't throw exception on an uppercase subfield identifier

Prevent Elasticsearch from throwing an exception when the mappings
include uppercase subfield identifier.

Though forbidden in MARC21, It doesn't seem to be the case for Unimarc
or ISO 2709 as a whole. It is also possible to define such a subfield
identifier in the Koha MARC Framework and in the Elasticsearch
mappings.

Test plan:

1. add a 995Z subfield identifier in Elasticsearch mappings
(Administration -> Search engine configuration)

2. run $KOHA_PATH/misc/search_tools/rebuild_elasticsearch.pl -v -d -b

3. see it throw: “Invalid MARC field expression: 995Z”

4. apply patch

5. rerun $KOHA_PATH/misc/search_tools/rebuild_elasticsearch.pl -v -d -b

6. sababa
Comment 2 Séverine Queune 2019-10-01 13:12:51 UTC
Created attachment 93365 [details] [review]
Bug 23671: Elasticsearch shouldn't throw exception on an uppercase subfield identifier

Prevent Elasticsearch from throwing an exception when the mappings
include uppercase subfield identifier.

Though forbidden in MARC21, It doesn't seem to be the case for Unimarc
or ISO 2709 as a whole. It is also possible to define such a subfield
identifier in the Koha MARC Framework and in the Elasticsearch
mappings.

Test plan:

1. add a 995Z subfield identifier in Elasticsearch mappings
(Administration -> Search engine configuration)

2. run $KOHA_PATH/misc/search_tools/rebuild_elasticsearch.pl -v -d -b

3. see it throw: “Invalid MARC field expression: 995Z”

4. apply patch

5. rerun $KOHA_PATH/misc/search_tools/rebuild_elasticsearch.pl -v -d -b

6. sababa

Signed-off-by: Séverine QUEUNE <severine.queune@bulac.fr>
Comment 3 Nick Clemens 2019-10-04 10:01:53 UTC
Created attachment 93730 [details] [review]
Bug 23671: Elasticsearch shouldn't throw exception on an uppercase subfield identifier

Prevent Elasticsearch from throwing an exception when the mappings
include uppercase subfield identifier.

Though forbidden in MARC21, It doesn't seem to be the case for Unimarc
or ISO 2709 as a whole. It is also possible to define such a subfield
identifier in the Koha MARC Framework and in the Elasticsearch
mappings.

Test plan:

1. add a 995Z subfield identifier in Elasticsearch mappings
(Administration -> Search engine configuration)

2. run $KOHA_PATH/misc/search_tools/rebuild_elasticsearch.pl -v -d -b

3. see it throw: “Invalid MARC field expression: 995Z”

4. apply patch

5. rerun $KOHA_PATH/misc/search_tools/rebuild_elasticsearch.pl -v -d -b

6. sababa

Signed-off-by: Séverine QUEUNE <severine.queune@bulac.fr>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Comment 4 Martin Renvoize 2019-10-07 11:58:30 UTC
Nice work!

Pushed to master for 19.11.00