Bug 27139

Summary: Search errors are incorrectly shown to users as no results found
Product: Koha Reporter: David Cook <dcook>
Component: SearchingAssignee: Bugs List <koha-bugs>
Status: NEW --- QA Contact: Testopia <testopia>
Severity: enhancement    
Priority: P5 - low CC: fridolin.somers, jonathan.druart
Version: Main   
Hardware: All   
OS: All   
See Also: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27291
Change sponsored?: --- Patch complexity: ---
Documentation contact: Documentation submission:
Text to go in the release notes:
Version(s) released in:
Circulation function:

Description David Cook 2020-12-03 01:47:05 UTC
When there is an error submitting a query to Zebra or Elasticsearch, the user should be informed that their query could not be completed due to an error with their search.
Comment 1 David Cook 2020-12-03 02:08:41 UTC
Related note:

If you're using Zebra, you have QueryAutoTruncate enabled, and QueryWeightFields enabled, you'll have fairly simple query building happening. It leads to the following situation:

Search:
"Emperor penguin (Aptenodytes forsteri) foraging ecology" 
Query:
@attrset Bib-1 @attr 1=1016 @attr 4=6 @attr 5=1 "Emperor penguin (Aptenodytes forsteri) foraging ecology"

Search:
Emperor penguin (Aptenodytes forsteri) foraging ecology
Query:
ZOOM error 10014 "CCL parsing error" (addinfo: "Operator expected") from diag-set 'ZOOM'

Search:
Emperor penguin \(Aptenodytes forsteri\) foraging ecology
Query:
@attrset Bib-1 @attr 1=1016 @attr 4=6 @attr 5=1 "Emperor penguin (Aptenodytes forsteri) foraging ecology"

ZOOM::Query::CCL2RPN returns an exception with code, addinfo, message, and diagset fields:
$VAR1 = bless( {
                 'code' => 10014,
                 'addinfo' => 'Operator expected',
                 'message' => 'CCL parsing error',
                 'diagset' => 'ZOOM'
               }, 'ZOOM::Exception' );

We would need to return this exception or derived data from C4::Search::getRecords() and then from Koha::SearchEngine::Zebra::Search::search_compat().
Comment 2 David Cook 2020-12-15 22:52:04 UTC
That said, I don't think any search system would ever say "Sorry, your query contains a syntax error. Try again."

So probably this is more of a "feature" than a "bug" that we don't tell the user that their search couldn't be processed.

It's a shame that we don't have a mechanism for storing searches that fail, so that we can analyze them and work to avoid their problems in the future...

While we might not think of bad search queries as "bad data", perhaps we should and leverage ideas regarding "bad data" such as "automated data quality monitoring and error detection".

I'll have to put some more thought into this, but I think that there's a seed there of something.
Comment 3 Fridolin Somers 2020-12-16 08:28:55 UTC
I like the idea
Comment 4 David Cook 2020-12-17 01:53:17 UTC
(In reply to Fridolin Somers from comment #3)
> I like the idea

I've done more thinking this morning, and now I'm starting to think about just monitoring Zebra better. Possibly putting together a Koha plugin to act as a Prometheus exporter, so it would scan Zebra logs for errors and then let Prometheus pull that data into itself. 

It could then be interrogated in Prometheus, and we as developers could then track back issues and improve Koha's Zebra integration as a result.
Comment 5 David Cook 2020-12-21 23:19:45 UTC
*** Bug 21287 has been marked as a duplicate of this bug. ***
Comment 6 David Cook 2020-12-22 00:25:01 UTC
Frido: Bug 27291 may be interesting to you. I think that change would make it a lot easier to monitor the Zebra logs...
Comment 7 Fridolin Somers 2020-12-22 09:21:19 UTC
(In reply to David Cook from comment #6)
> Frido: Bug 27291 may be interesting to you. I think that change would make
> it a lot easier to monitor the Zebra logs...

Oooo nice
Comment 8 Jonathan Druart 2021-01-21 10:11:03 UTC
It works for ES (OPAC and staff interfaces):

"""
No results found!

You did not specify any search criteria.

Error:
Unable to perform your search. Please try again. 
"""

The "You did not specify any search criteria." part is incorrect, but the user is informed something wrong happened.
Comment 9 David Cook 2021-01-22 01:24:38 UTC
(In reply to Jonathan Druart from comment #8)
> The "You did not specify any search criteria." part is incorrect, but the
> user is informed something wrong happened.

Hmm, is that something that we want to concede to users? 

If so, then adding feature parity with ES for Zebra isn't a bad idea, although one of these days I will over to ES...
Comment 10 Jonathan Druart 2021-01-22 08:46:57 UTC
(In reply to David Cook from comment #9)
> (In reply to Jonathan Druart from comment #8)
> > The "You did not specify any search criteria." part is incorrect, but the
> > user is informed something wrong happened.
> 
> Hmm, is that something that we want to concede to users? 

Yes, I think so, and it's what you wanted on comment 0 if I understand correctly :)
Comment 11 David Cook 2021-01-24 23:00:25 UTC
(In reply to Jonathan Druart from comment #10)
> (In reply to David Cook from comment #9)
> > (In reply to Jonathan Druart from comment #8)
> > > The "You did not specify any search criteria." part is incorrect, but the
> > > user is informed something wrong happened.
> > 
> > Hmm, is that something that we want to concede to users? 
> 
> Yes, I think so, and it's what you wanted on comment 0 if I understand
> correctly :)

True although I changed my mind later heh