Summary: | Elasticsearch - authority indexing depends on mapping order | ||
---|---|---|---|
Product: | Koha | Reporter: | Nick Clemens (kidclamp) <nick> |
Component: | Searching - Elasticsearch | Assignee: | Nick Clemens (kidclamp) <nick> |
Status: | CLOSED FIXED | QA Contact: | |
Severity: | major | ||
Priority: | P5 - low | CC: | 1joynelson, andrew, ere.maijala, jonathan.druart, lucas, martin.renvoize, mkstephens |
Version: | Main | ||
Hardware: | All | ||
OS: | All | ||
Change sponsored?: | --- | Patch complexity: | --- |
Documentation contact: | Documentation submission: | ||
Text to go in the release notes: | Version(s) released in: |
20.05.00, 19.11.06
|
|
Circulation function: | |||
Attachments: |
Bug 25050: Unit tests
Bug 25050: Adjust marc_records_to_documents to search entire mapping array Mappings that uncovered the bug Bug 25050: Unit tests Bug 25050: Adjust marc_records_to_documents to search entire mapping array Bug 25050: Unit tests Bug 25050: Adjust marc_records_to_documents to search entire mapping array |
Description
Nick Clemens (kidclamp)
2020-04-03 12:12:30 UTC
Created attachment 102346 [details] [review] Bug 25050: Unit tests This patch simplay alters the data we use for the tests, doing so causes them to fail To test: 1 - Apply only this patch 2 - prove -v t/Koha/SearchEngine/Elasticsearch.t 3 - It fails! 4 - Apply next patch 5 - It passes! Created attachment 102347 [details] [review] Bug 25050: Adjust marc_records_to_documents to search entire mapping array There was an assumption in the ES code that match-heading mappings will appear in a specified portion of the mappings array. Certain mappings setups will not meet this assumption. We need to move our searching up one level The key seems to be having a mapping for a complete field, say 150, in both the match-heading and another field as well as having mappings for ungrouped fields like 150a 150ab etc. The unit test coverage should be sufficient for testing Created attachment 102348 [details]
Mappings that uncovered the bug
You can test with this mapping file.
Donwload and save the file.
Alter your koha-conf.xml to point to this file.
Reload your koha mappings.
Reset all the things
Reindex ES
Check an auth record:
curl -XGET 'es:9200/koha_kohadev_authorities/data/568?pretty' | grep -A 5 match-heading
Note match heading contains the terms, but not a formatted heading string
Apply patch
Reindex ES
Check an auth record
You should now have a string with 'subdiv' terms e.g.:
Authors, Irish chronologicalsubdiv 20th century formsubdiv Biography
Created attachment 102356 [details] [review] Bug 25050: Unit tests This patch simplay alters the data we use for the tests, doing so causes them to fail To test: 1 - Apply only this patch 2 - prove -v t/Koha/SearchEngine/Elasticsearch.t 3 - It fails! 4 - Apply next patch 5 - It passes! Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com> Created attachment 102357 [details] [review] Bug 25050: Adjust marc_records_to_documents to search entire mapping array There was an assumption in the ES code that match-heading mappings will appear in a specified portion of the mappings array. Certain mappings setups will not meet this assumption. We need to move our searching up one level The key seems to be having a mapping for a complete field, say 150, in both the match-heading and another field as well as having mappings for ungrouped fields like 150a 150ab etc. The unit test coverage should be sufficient for testing Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com> Created attachment 102683 [details] [review] Bug 25050: Unit tests This patch simplay alters the data we use for the tests, doing so causes them to fail To test: 1 - Apply only this patch 2 - prove -v t/Koha/SearchEngine/Elasticsearch.t 3 - It fails! 4 - Apply next patch 5 - It passes! Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com> Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org> Created attachment 102684 [details] [review] Bug 25050: Adjust marc_records_to_documents to search entire mapping array There was an assumption in the ES code that match-heading mappings will appear in a specified portion of the mappings array. Certain mappings setups will not meet this assumption. We need to move our searching up one level The key seems to be having a mapping for a complete field, say 150, in both the match-heading and another field as well as having mappings for ungrouped fields like 150a 150ab etc. The unit test coverage should be sufficient for testing Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com> Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org> Nice work everyone! Pushed to master for 20.05 backported to 19.11.x for 19.11.06 doesnt apply clean to 19.05.x, if needed please rebase |