If you're in the advanced search in the staff client, and if you're using the QueryWeightFields system preference, and if you're searching with "Corporate-name", you'll generate Zebra errors. There are a few things contributing to this problem. One is that Corporate-name is only indexed into the "word" register and not the "phrase" register. This becomes a problem with QueryWeightFields, because of the last conditional block in C4::Search::_build_weighted_query() which rewrites the search query to include $index,ext,r1, which forces a search of the "phrase" register, which causes Zebra to explode (when it doesn't use the 14=1 special attribute).
For example: 11:46:06-30/08 zebrasrv(44) [request] Search biblios ERROR 114 1 1+0 RPN @attrset Bib-1 @and @or @or @attr 1=Corporate-name @attr 4=1 @attr 6=3 @attr 9=32 @attr 2=102 corruption @attr 1=Corporate-name @attr 4=1 @attr 9=26 @attr 2=102 corruption @attr 1=Corporate-name @attr 4=6 @attr 9=20 @attr 2=102 corruption @attr 1=homebranch CPL If you try that same search in yaz-client, you'll get something like: Z> f @attrset Bib-1 @and @or @or @attr 1=Corporate-name @attr 4=1 @attr 6=3 @attr 9=32 @attr 2=102 corruption @attr 1=Corporate-name @attr 4=1 @attr 9=26 @attr 2=102 corruption @attr 1=Corporate-name @attr 4=6 @attr 9=20 @attr 2=102 corruption @attr 1=homebranch CPL Sent searchRequest. Received SearchResponse. Search was a bloomin' failure. Number of hits: 0, setno 1 Result Set Status: none records returned: 0 Diagnostic message(s) from database: [114] Unsupported Use attribute -- v2 addinfo 'Corporate-name' Elapsed: 0.006073 You can get some insight from this wiki entry: https://wiki.koha-community.org/wiki/Troubleshooting_Zebra#What_is_an_Unsupported_Use_attribute.3F (I wrote this years ago and it needs work but it's mostly on point.) So it's an issue with the index not belonging in a particular register. As per my earlier comment and based on the search query, it's the "phrase" register it's missing from. The interesting thing is that if we remove the @attr 6=3 from the above query, the query returns successfully. That's because the "phrase" register is only used for complete fields (ie for queries that have a @attr 6=3 attribute). You can see this in Table 5.13 at https://software.indexdata.com/zebra/doc/querymodel-zebra.html. Mind blown.
Now the question is do we put Corporate-name in the phrase register in the Zebra indexing XSLT... ...or do we re-think C4::Search::_build_weighted_query() because it's probably causing this exact same problem in lots of other contexts.
Or do we not care because we're just trying to move forward with Elasticsearch?
(In reply to David Cook from comment #2) > Now the question is do we put Corporate-name in the phrase register in the > Zebra indexing XSLT... > > ...or do we re-think C4::Search::_build_weighted_query() because it's > probably causing this exact same problem in lots of other contexts. Indeed, its a tricky error. We have the same issue in UNIMARC. I think we may change etc/zebradb/xsl/koha-indexdefs-to-zebra.xsl to always add phrase indexing on a words-list index (like Corporate-name:w)
Created attachment 121326 [details] [review] Bug 21286: Add Corporate-name as phrase to zebra indexes When using Zebra for searching, Koha performs a number of searches in order to improve relevancy. This means that even for 'wordlist' search, we perform a phrase search. When selecting 'Corporate-name' as an index, this expansion of the search causes errors and fails the search We can fix this for 'Corporate-name' searches by adding a phrase index To test: 1 - Edit koha-conf.xml and uncomment the zebra debug line and add 'request' to the list 2 - Restart all 3 - tail -f /var/log/koha/kohadev/zebra-output.log 4 - Edit a record to add a 110 field e.g. 'House plants' 5 - Enable syspref IntranetCatalogSearchPulldown 6 - Search for 'Corporate name' and term 'House plants' 7 - No results 8 - View the log, see 'ERROR' and full search terms listed 9 - Apply patch 10 - copy the zebra files to the production instance: cp etc/zebradb/marc_defs/marc21/biblios/biblio-koha-indexdefs.xml /etc/koha/zebradb/marc_defs/marc21/biblios/biblio-koha-indexdefs.xml cp etc/zebradb/marc_defs/marc21/biblios/biblio-zebra-indexdefs.xsl /etc/koha/zebradb/marc_defs/marc21/biblios/biblio-zebra-indexdefs.xsl 11 - restart all 12 - rebuild: sudo koha-rebuild-zebra -v -f kohadev 13 - Repeat search 14 - Success!
I attempted to test this, but I couldn't recreate the error(steps 4-8) - I could find the record with the corporate name in it (I added a unique name not used anywhere else). The QueryWeightFields system preference is enabled. I changed the dropdown list to 'Corporate name' and also tried with 'Corporate name as a phrase'. Is there something else I need to do to try and recreate the issue? Here is the Zebra log entry for the request: 01:28:46-30/05 zebrasrv(54) [request] Search biblios OK 1 1 1+0 RPN @attrset Bib-1 @attr 1=Corporate-name @attr 4=6 @attr 5=1 @attr 2=102 "xanadu mftc" 01:28:46-30/05 zebrasrv(54) [request] Present OK - 1 1+1 01:28:46-30/05 zebrasrv(54) [request] Present OK - 1 1+1 01:28:46-30/05 zebrasrv(54) [request] Present OK - 1 1+1 01:28:46-30/05 zebrasrv(54) [request] Present OK - 1 1+1 01:28:46-30/05 zebrasrv(54) [request] Present OK - 1 1+1 01:28:46-30/05 zebrasrv(54) [request] Present OK - 1 1+1 01:28:46-30/05 zebrasrv(54) [request] Present OK - 1 1+1 01:28:46-30/05 zebrasrv(54) [request] Present OK - 1 1+1 01:28:46-30/05 zebrasrv(54) [request] Present OK - 1 1+1 01:28:46-30/05 zebrasrv(54) [request] Present OK - 1 1+1 01:28:46-30/05 zebrasrv(55) [request] Auth idPass kohauser - 01:28:46-30/05 zebrasrv(55) [request] Init OK - ID:81 Name:ZOOM-C/YAZ Version:5.27.1 872b6f92a024bb53bc1c11dfeccd47f065f98238 01:28:46-30/05 zebrasrv(55) [request] Search biblios OK 0 1 1+0 RPN @attrset Bib-1 @attr 1=1033 "Programming Perl /" Testing notes (koha-testing-docker): - Enabled Zebra logging: . edit /etc/koha/sites/kohadev/koha-conf.xml . find zebra_loglevels setting . uncomment and add request - Added "xanadu mftc" as an authority record and linked to a record
(In reply to David Nind from comment #6) > I attempted to test this, but I couldn't recreate the error(steps 4-8) - I > could find the record with the corporate name in it (I added a unique name > not used anywhere else). > Try disabling the syspref 'QueryAutoTruncate' - that seems to be a key to triggering this
> Try disabling the syspref 'QueryAutoTruncate' - that seems to be a key to > triggering this Thanks Nick! That worked, sign off on the way..
Created attachment 121554 [details] [review] Bug 21286: Add Corporate-name as phrase to zebra indexes When using Zebra for searching, Koha performs a number of searches in order to improve relevancy. This means that even for 'wordlist' search, we perform a phrase search. When selecting 'Corporate-name' as an index, this expansion of the search causes errors and fails the search We can fix this for 'Corporate-name' searches by adding a phrase index To test: 1 - Edit koha-conf.xml and uncomment the zebra debug line and add 'request' to the list 2 - Restart all 3 - tail -f /var/log/koha/kohadev/zebra-output.log 4 - Edit a record to add a 110 field e.g. 'House plants' 5 - Enable syspref IntranetCatalogSearchPulldown 6 - Search for 'Corporate name' and term 'House plants' 7 - No results 8 - View the log, see 'ERROR' and full search terms listed 9 - Apply patch 10 - copy the zebra files to the production instance: cp etc/zebradb/marc_defs/marc21/biblios/biblio-koha-indexdefs.xml /etc/koha/zebradb/marc_defs/marc21/biblios/biblio-koha-indexdefs.xml cp etc/zebradb/marc_defs/marc21/biblios/biblio-zebra-indexdefs.xsl /etc/koha/zebradb/marc_defs/marc21/biblios/biblio-zebra-indexdefs.xsl 11 - restart all 12 - rebuild: sudo koha-rebuild-zebra -v -f kohadev 13 - Repeat search 14 - Success! Signed-off-by: David Nind <david@davidnind.com>
Created attachment 121567 [details] [review] Bug 21286: Add Corporate-name as phrase to zebra indexes When using Zebra for searching, Koha performs a number of searches in order to improve relevancy. This means that even for 'wordlist' search, we perform a phrase search. When selecting 'Corporate-name' as an index, this expansion of the search causes errors and fails the search We can fix this for 'Corporate-name' searches by adding a phrase index To test: 1 - Edit koha-conf.xml and uncomment the zebra debug line and add 'request' to the list 2 - Restart all 3 - tail -f /var/log/koha/kohadev/zebra-output.log 4 - Edit a record to add a 110 field e.g. 'House plants' 5 - Enable syspref IntranetCatalogSearchPulldown 6 - Search for 'Corporate name' and term 'House plants' 7 - No results 8 - View the log, see 'ERROR' and full search terms listed 9 - Apply patch 10 - copy the zebra files to the production instance: cp etc/zebradb/marc_defs/marc21/biblios/biblio-koha-indexdefs.xml /etc/koha/zebradb/marc_defs/marc21/biblios/biblio-koha-indexdefs.xml cp etc/zebradb/marc_defs/marc21/biblios/biblio-zebra-indexdefs.xsl /etc/koha/zebradb/marc_defs/marc21/biblios/biblio-zebra-indexdefs.xsl 11 - restart all 12 - rebuild: sudo koha-rebuild-zebra -v -f kohadev 13 - Repeat search 14 - Success! Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Pushed to master for 21.11, thanks to everybody involved!
Pushed to 21.05.x for 21.05.02
Knowing that, we may add in koha-indexdefs-to-zebra.xsl the phrase form if missing
Pushed to 20.11.x for 20.11.08
Not backported to oldoldstable (20.05.x). Feel free to ask if it's needed.