Bug 18163

Summary: Elasticsearch advanced search by item type not working with item-level_itypes turned off (biblio record)
Product: Koha Reporter: Pedro Amorim <pjamorim91>
Component: Searching - ElasticsearchAssignee: Nick Clemens <nick>
Status: CLOSED FIXED QA Contact:
Severity: minor    
Priority: P5 - low CC: bibliwho, jonathan.druart, josef.moravec
Version: unspecified   
Hardware: All   
OS: Linux   
Change sponsored?: --- Patch complexity: Trivial patch
Documentation contact: Documentation submission:
Text to go in the release notes:
Version(s) released in:
Bug Depends on: 12478    
Bug Blocks:    
Attachments: Bug 18163: Unit tests
Bug 18163: Pass through limits in ES
Bug 18163: Unit tests
Bug 18163: Pass through limits in ES

Description Pedro Amorim 2017-02-23 11:14:44 UTC
After installing and indexing Elasticsearch, a search for itemtype is not working correctly. The reason for this I think is that Marc21 uses the index itype whereas Unimarc uses the index itemtype.
The index mapped in mappings.yml is itype, and a search for item type in unimarc will search on itemtype index, which has nothing mapped to.

Furthermore, trying to change the index from itype to itemtype in the mappings panel for elasticsearch will trigger the following error:

Can't call method "label" on an undefined value at /usr/share/koha/intranet/cgi-bin/admin/searchengine/elasticsearch/mappings.pl line 69
Comment 1 Pedro Amorim 2017-02-23 11:50:28 UTC
I added the itemtype index in mappings.yaml and reset it. Still not working.
The itype is indeed being used in unimarc and working properly but only from the facets.
Searching from the advanced search still returns no results and the search is being conducted with "itemtype,phr:TYPE". If I manually change this to "itemtype:TYPE" it's returning the records with item type TYPE correctly. Can the problem be in the phrase?
Comment 2 Pedro Amorim 2017-02-23 12:07:21 UTC
Turns out the the difference between itype and itemtype is related to the item-level_itypes preference, which i had it set to biblio record instead of item.

I changed the preference back to item-level and now the search is working. However, upon indexing, the following message is thrown:
"item-level_itypes set but no itemtype set for item (ID) at /usr/share/koha/lib/Koha/Schema/Result/Item.pm line 698. "
Which is confusing because the item type info is in biblioitems.itemtype for the items and also the search is apparently working fine.
Comment 3 Pedro Amorim 2017-02-23 12:17:15 UTC
The reason the search is throwing the warning:
"item-level_itypes set but no itemtype set for item (ID) at /usr/share/koha/lib/Koha/Schema/Result/Item.pm line 698. " is because the items.itype are indeed NULL (no info at the time of import). And the reason the search is working is because it falls back to the biblioitems.itemtype after throwing the error:
https://github.com/Koha-Community/Koha/blob/4f178fad53fb748bd1ccf67680e02411918c9584/Koha/Schema/Result/Item.pm#L700

With all that, the bug remains, the advanced search by item type will return no results because it's searching as "itemtype, phr". "itemtype" alone will return the results correctly.
Comment 4 Nick Clemens 2018-05-25 13:17:20 UTC
Created attachment 75564 [details] [review]
Bug 18163: Unit tests

prove -v t/db_dependent/Koha/SearchEngine/Elasticsearch/QueryBuilder.t
Comment 5 Nick Clemens 2018-05-25 13:17:23 UTC
Created attachment 75565 [details] [review]
Bug 18163: Pass through limits in ES

Before these patches we only allow a small subset of indexes to be used
as limits - if we are going to let users add new fields eventually we
probably don't want to strip out terms we didn't plan for.

To test:
1 - Enable ES
2 - Using sample kohadevbox data try a search like this (or adjust as necessary):
http://localhost:8080/cgi-bin/koha/opac-search.pl?idx=kw&idx=kw&idx=kw&limit=subject%2Cphr%3Astreet&sort_by=relevance&do=Search
3 - Note nothing is found
4 - Apply patches
5 - Repeat search
6 - It works!
Comment 6 Cab Vinton 2018-08-27 09:29:21 UTC
Unable to replicate in Biblibre sandboxes 11 (this patch) & 12 (master), w/ database 2 (UNIMARC).

Tried searching by subject and subject phrase, & both with and without an item type limit:
opac-search.pl?idx=su%2Cphr&q=rugby&op=and&idx=kw&q=&op=and&idx=kw&q=&do=Search&limit=mc-itype%2Cphr%3ALIVR

Search returned the same 2 results in both sandboxes.
Comment 7 Josef Moravec 2019-03-01 09:20:19 UTC
Created attachment 85891 [details] [review]
Bug 18163: Unit tests

prove -v t/db_dependent/Koha/SearchEngine/Elasticsearch/QueryBuilder.t
Comment 8 Josef Moravec 2019-03-01 09:20:22 UTC
Created attachment 85892 [details] [review]
Bug 18163: Pass through limits in ES

Before these patches we only allow a small subset of indexes to be used
as limits - if we are going to let users add new fields eventually we
probably don't want to strip out terms we didn't plan for.

To test:
1 - Enable ES
2 - Using sample kohadevbox data try a search like this (or adjust as necessary):
http://localhost:8080/cgi-bin/koha/opac-search.pl?idx=kw&idx=kw&idx=kw&limit=subject%2Cphr%3Astreet&sort_by=relevance&do=Search
3 - Note nothing is found
4 - Apply patches
5 - Repeat search
6 - It works!
Comment 9 Josef Moravec 2019-03-01 09:21:06 UTC
Hi Nick, nice enhancement, but the test does not cover the change - it passes with and without the second patch.

I am attaching rebased patches, as there were some minor conflicts.
Comment 10 Nick Clemens 2019-04-22 13:46:50 UTC
Fixed by bug 19575 commit 6d53910f0ca05ae41b0d10ce6f3eb274efd3aca4