Bug 25957 - Elasticsearch 5.X - empty subfields cause error on suggestible fields
Summary: Elasticsearch 5.X - empty subfields cause error on suggestible fields
Status: CLOSED FIXED
Alias: None
Product: Koha
Classification: Unclassified
Component: Searching - Elasticsearch (show other bugs)
Version: Main
Hardware: All All
: P5 - low normal
Assignee: Nick Clemens (kidclamp)
QA Contact:
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2020-07-08 17:29 UTC by Nick Clemens (kidclamp)
Modified: 2021-06-14 21:31 UTC (History)
4 users (show)

See Also:
Change sponsored?: ---
Patch complexity: ---
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:
20.11.00, 20.05.05
Circulation function:


Attachments
Bug 25957: Unit test (2.13 KB, patch)
2020-07-08 17:31 UTC, Nick Clemens (kidclamp)
Details | Diff | Splinter Review
Bug 25957: Don't push blank field values to ES document (1.87 KB, patch)
2020-07-08 17:31 UTC, Nick Clemens (kidclamp)
Details | Diff | Splinter Review
Bug 25957: Unit test (2.19 KB, patch)
2020-09-28 15:31 UTC, Nick Clemens (kidclamp)
Details | Diff | Splinter Review
Bug 25957: Don't push blank field values to ES document (1.98 KB, patch)
2020-09-28 15:31 UTC, Nick Clemens (kidclamp)
Details | Diff | Splinter Review
Bug 25957: Unit test (2.24 KB, patch)
2020-09-28 16:41 UTC, Andrew Fuerste-Henry
Details | Diff | Splinter Review
Bug 25957: Don't push blank field values to ES document (2.03 KB, patch)
2020-09-28 16:41 UTC, Andrew Fuerste-Henry
Details | Diff | Splinter Review
Bug 25957: Unit test (2.31 KB, patch)
2020-10-05 14:56 UTC, Martin Renvoize (ashimema)
Details | Diff | Splinter Review
Bug 25957: Don't push blank field values to ES document (2.09 KB, patch)
2020-10-05 14:56 UTC, Martin Renvoize (ashimema)
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Nick Clemens (kidclamp) 2020-07-08 17:29:26 UTC
To recreate:
1 - Load the sample DB or edit a record (using advanced cataloging editor) to have a blank subfield in a field that is indexed as suggestible
2 - For example 'author' / 100a
    100 _ _ ‡a
3 - Index that record into Elasticsearch 5.X:
    perl misc/search_tools/rebuild_elasticsearch.pl  -v -bn 115 -b -d
4 - Note error 'value must have length > 0'
5 - Edit mappings to set author 100a not suggestible
6 - perl misc/search_tools/rebuild_elasticsearch.pl  -v -bn 115 -b -d
7 - Success
Comment 1 Nick Clemens (kidclamp) 2020-07-08 17:31:40 UTC
Created attachment 106660 [details] [review]
Bug 25957: Unit test
Comment 2 Nick Clemens (kidclamp) 2020-07-08 17:31:43 UTC
Created attachment 106661 [details] [review]
Bug 25957: Don't push blank field values to ES document

To test:
 1 - Load the sample DB or edit a record (using advanced cataloging editor) to have a blank subfield in a field that is indexed as suggestible
 2 - For example 'author' / 100a
     100 _ _ ‡a
 3 - Index that record into Elasticsearch 5.X:
     perl misc/search_tools/rebuild_elasticsearch.pl  -v -bn 115 -b -d
 4 - Note error 'value must have length > 0'
 5 - Edit mappings to set author 100a not suggestible
 6 - perl misc/search_tools/rebuild_elasticsearch.pl  -v -bn 115 -b -d
 7 - Success
 8 - Set field to suggestible again
 9 - Apply patch
10 - perl misc/search_tools/rebuild_elasticsearch.pl  -v -bn 115 -b -d
11 - Success!
Comment 3 Nick Clemens (kidclamp) 2020-09-28 15:31:05 UTC
Created attachment 110877 [details] [review]
Bug 25957: Unit test
Comment 4 Nick Clemens (kidclamp) 2020-09-28 15:31:17 UTC
Created attachment 110878 [details] [review]
Bug 25957: Don't push blank field values to ES document

To test:
 1 - Load the sample DB or edit a record (using advanced cataloging editor) to have a blank subfield in a field that is indexed as suggestible
 2 - For example 'author' / 100a
     100 _ _ ‡a
 3 - Index that record into Elasticsearch 5.X:
     perl misc/search_tools/rebuild_elasticsearch.pl  -v -bn 115 -b -d
 4 - Note error 'value must have length > 0'
 5 - Edit mappings to set author 100a not suggestible
 6 - perl misc/search_tools/rebuild_elasticsearch.pl  -v -bn 115 -b -d
 7 - Success
 8 - Set field to suggestible again
 9 - Apply patch
10 - perl misc/search_tools/rebuild_elasticsearch.pl  -v -bn 115 -b -d
11 - Success!
Comment 5 Andrew Fuerste-Henry 2020-09-28 16:41:32 UTC
Created attachment 110880 [details] [review]
Bug 25957: Unit test

Signed-off-by: Bob Bennhoff <bbennhoff@clicweb.org>
Comment 6 Andrew Fuerste-Henry 2020-09-28 16:41:36 UTC
Created attachment 110881 [details] [review]
Bug 25957: Don't push blank field values to ES document

To test:
 1 - Load the sample DB or edit a record (using advanced cataloging editor) to have a blank subfield in a field that is indexed as suggestible
 2 - For example 'author' / 100a
     100 _ _ ‡a
 3 - Index that record into Elasticsearch 5.X:
     perl misc/search_tools/rebuild_elasticsearch.pl  -v -bn 115 -b -d
 4 - Note error 'value must have length > 0'
 5 - Edit mappings to set author 100a not suggestible
 6 - perl misc/search_tools/rebuild_elasticsearch.pl  -v -bn 115 -b -d
 7 - Success
 8 - Set field to suggestible again
 9 - Apply patch
10 - perl misc/search_tools/rebuild_elasticsearch.pl  -v -bn 115 -b -d
11 - Success!

Signed-off-by: Bob Bennhoff <bbennhoff@clicweb.org>
Comment 7 Martin Renvoize (ashimema) 2020-10-05 14:56:55 UTC
Created attachment 111240 [details] [review]
Bug 25957: Unit test

Signed-off-by: Bob Bennhoff <bbennhoff@clicweb.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 8 Martin Renvoize (ashimema) 2020-10-05 14:56:58 UTC
Created attachment 111241 [details] [review]
Bug 25957: Don't push blank field values to ES document

To test:
 1 - Load the sample DB or edit a record (using advanced cataloging editor) to have a blank subfield in a field that is indexed as suggestible
 2 - For example 'author' / 100a
     100 _ _ ‡a
 3 - Index that record into Elasticsearch 5.X:
     perl misc/search_tools/rebuild_elasticsearch.pl  -v -bn 115 -b -d
 4 - Note error 'value must have length > 0'
 5 - Edit mappings to set author 100a not suggestible
 6 - perl misc/search_tools/rebuild_elasticsearch.pl  -v -bn 115 -b -d
 7 - Success
 8 - Set field to suggestible again
 9 - Apply patch
10 - perl misc/search_tools/rebuild_elasticsearch.pl  -v -bn 115 -b -d
11 - Success!

Signed-off-by: Bob Bennhoff <bbennhoff@clicweb.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 9 Martin Renvoize (ashimema) 2020-10-05 14:58:58 UTC
Works as expected, no regressions found.

Passing QA
Comment 10 Jonathan Druart 2020-10-08 09:37:52 UTC
Pushed to master for 20.11, thanks to everybody involved!
Comment 11 Lucas Gass (lukeg) 2020-10-20 19:57:57 UTC
backported to 20.05.x for 20.05.05
Comment 12 Aleisha Amohia 2020-10-27 04:34:52 UTC
doesn't apply cleanly to 19.11.x, please rebase if needed