Bug 18624

Summary: Software error when searching authorities in Elasticsearch - incorrect parameter "any" should be "all"
Product: Koha Reporter: david holoshka <david.holoshka>
Component: SearchingAssignee: Galen Charlton <gmcharlt>
Status: Needs Signoff --- QA Contact: Testopia <testopia>
Severity: critical    
Priority: P5 - low CC: nick
Version: master   
Hardware: PC   
OS: Linux   
Whiteboard:
Change sponsored?: --- Patch complexity: ---
Bot Control: --- When did the bot last check this:
Who signed the patch off: Text to go in the release notes:
Attachments: Bug 18624 - Run time errors when searching authorities with Elastic search 5.3

Description david holoshka 2017-05-18 07:40:44 UTC
Searching for a known authority entry (eg. Bear or Meyer) results in either the following runtime errors or no result (sort none).  Sort on finds nothing

Header search only $a:
Software error:

Invalid marclist field provided: all at /usr/local/koha/Koha/SearchEngine/Elasticsearch/QueryBuilder.pm line 433.
	Koha::SearchEngine::Elasticsearch::QueryBuilder::build_authorities_query_compat(Koha::SearchEngine::Elasticsearch::QueryBuilder=HASH(0x90e2110),
 ARRAY(0x90d4630), ARRAY(0x90d4660), ARRAY(0x914bf30), ARRAY(0x912f6b0), ARRAY(0x912f0f8), "", "") called at /usr/local/koha/authorities/authorities-home.pl line 93


Search Main Heading :

Software error:

Can't use string ("9932563") as an ARRAY ref while "strict refs" in use at /usr/local/koha/Koha/SearchEngine/Elasticsearch/Search.pm line 202.


auth index entry in elastic search:

 {
        "_index" : "koha_ulibrary_authorities",
        "_type" : "data",
        "_id" : "1756",
        "_score" : 1.0,
        "_source" : {
          "Personal-name-see-also-from" : null,
          "Meeting-name-heading" : null,
          "Personal-name" : [
            "Bear, Roberta Meyer"
          ],
          "Corporate-name-see-also-from" : null,
          "Heading-use-series-added-entry" : " ",
          "es_id" : "1756",
          "See-also-from" : null,
          "Meeting-name-see-also-from" : null,
          "LC-card-number" : null,
          "record" : [
            [
              "LDR",
              null,
              null,
              "_",
              "00262nz  a2200121o  4500"
            ],
            [
              "001",
              null,
              null,
              "_",
              "11072"
            ],
            [
              "003",
              null,
              null,
              "_",
              "LDU"
            ],
            [
              "005",
              null,
              null,
              "_",
              "20170405133903.0"
            ],
            [
              "008",
              null,
              null,
              "_",
              "170405930116   a    aabn           a aaa"
            ],
            [
              "040",
              " ",
              " ",
              "a",
              "L",
              "c",
              "L"
            ],
            [
              "100",
              "1",
              " ",
              "a",
              "Bear, Roberta Meyer"
            ],
            [
              "400",
              "1",
              " ",
              "a",
              "Meyer Bear, Roberta"
            ],
            [
              "942",
              " ",
              " ",
              "a",
              "PERSO_NAME"
            ],
            [
              "999",
              " ",
              " ",
              "c",
              "1756",
              "d",
              "1756"
            ]
          ],
          "Match-heading-see-from" : null,
          "record-source" : [
            "L",
            "L"
          ],
          "Heading-use-main-or-added-entry" : " ",
          "Meeting-name-see-from" : null,
          "Personal-name-heading" : [
            "Bear, Roberta Meyer"
          ],
          "Heading-use-subject-added-entry" : "a",
          "Personal-name-see-from" : [
            "Meyer Bear, Roberta"
          ],
          "Local-number" : [
            "11072"
          ],
          "Corporate-name-see-from" : null,
          "Match" : null,
          "authtype" : [
            "PERSO_NAME"
          ],
          "Kind-of-record" : "1",
          "Match-heading" : [
            "Bear, Roberta Meyer"
          ],
          "Descriptive-cataloging-rules" : "1",
          "See-from" : null,
          "Heading" : [
            "Bear, Roberta Meyer"
          ],
          "Heading-Main" : null,
          "Subject-heading-thesaurus" : "6",
          "Meeting-name" : null
        }
      },

We are using the default field mapping.
Koha running on Debian, I have applied a patch not yet on main (Bug 18434 - Elasticsearch indexing broken with newer catmandu version ) to make the biblio search work. 
Mysql

ES 5.3  (it seems to be working fine) on oracle java 8

Marc 21

All perl catmandu modules are the latest from cpanm.
Comment 1 Nick Clemens 2017-06-16 13:56:46 UTC
Created attachment 64385 [details] [review]
Bug 18624 - Run time errors when searching authorities with Elastic search 5.3

It seems we have a syntax mismatch - any vs all

all seems to the be list we expect so updating code that way

To test:
1 - Enabled Elasticsearch
2 - Index some authorities
3 - Perform a 'Search entire record' search
4 - Internal server error (
        Invalid marclist field provided: all at
        /usr/local/koha/Koha/SearchEngine/Elasticsearch/QueryBuilder.pm
        line 433.
        )
5 - Apply patch
6 - Search should work
Comment 2 Nick Clemens 2017-06-16 14:00:22 UTC
This will take care of the first error at least.

I can't recreate the "Can't use string ("9932563") as an ARRAY ref" - can you file that as a new bug with specific instructions for that issue?

There are additional things you mention here, they should probably each go on a separate bug as well (no result, sort error, etc) it is easier to fix things one at a time :-)

I am also adding a followup to 18434 as wee need to append entries for facets and sorting as well - i will be looking at fixing the tests / diagnosing why jonathan has differing results asap.