Bug 1807 - search () give odd results with Zebra and ICU
Summary: search () give odd results with Zebra and ICU
Status: CLOSED WORKSFORME
Alias: None
Product: Koha
Classification: Unclassified
Component: Searching - Zebra (show other bugs)
Version: master
Hardware: All All
: P3 normal with 1 vote (vote)
Assignee: Galen Charlton
QA Contact:
URL:
Keywords:
Depends on:
Blocks: 9234
  Show dependency treegraph
 
Reported: 2008-01-30 11:50 UTC by Frédéric Demians
Modified: 2023-12-28 20:44 UTC (History)
22 users (show)

See Also:
Change sponsored?: ---
Patch complexity: Trivial patch
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:


Attachments
Patch to fix searching on () (1.47 KB, patch)
2009-12-12 08:17 UTC, Chris Cormack
Details | Diff | Splinter Review
Patch to stop : blocking searches (861 bytes, patch)
2009-12-12 09:08 UTC, Chris Cormack
Details | Diff | Splinter Review
Bug 1807: Remove parens from links to fix searches (8.27 KB, patch)
2015-01-29 20:21 UTC, Nicole C. Engard
Details | Diff | Splinter Review
[SIGNED OFF] Bug 1807: Remove parens from links to fix searches (8.32 KB, patch)
2015-01-31 16:32 UTC, Nick Clemens
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Chris Cormack 2010-05-21 00:34:49 UTC


---- Reported by frederic@tamil.fr 2008-01-30 23:50:43 ----

SEARCH            RESULT
black panthers    Some biblio records
black : panthers  Nothing



---- Additional Comments From henridamien@koha-fr.org 2008-02-01 00:43:30 ----

this is owed to the fact that no matter what is before, : or = in search boxes are considered as ccl commands :
here index black contains panther.

This is a bug in this case but is a feature for big search box




---- Additional Comments From frederic@tamil.fr 2008-02-01 01:30:14 ----

You're correct. Being able to send CCL queries directly to Zebra search
engine is a great feature for power users. But it doesn't make sense for
end user, especially in a Google-Like search box. End users search for books
typing title and subtitle separated by :.

A solution could be to allow : interpretation only in Advanced search when
Keyword index is selected. 



---- Additional Comments From henridamien@koha-fr.org 2008-03-15 15:40:56 ----

Moving criticity up Since this can cause ppl not to be confident in our search.
indeed, if they copy paste a title which contains : or = then they CANNOT find it !!!!



---- Additional Comments From jmf@liblime.com 2008-05-19 13:43:48 ----

The right way to fix this would be to assign a specific subset of ccl-style parameters such as ti: for title, su: for subject, and only treat those : as assignment operators, and the rest of the ':' and '=' as keyword terms. I'll take a look at what this will take to accomplish.



---- Additional Comments From paul.poulain@biblibre.com 2008-06-05 06:04:51 ----

couldn't we request that an explicit ccl query start by "ccl:"

so "title = bear" => search "title & bear"
and "ccl: title=bear" => CCL search on title for value "bear"

I think that a poweruser entering an equation can understand he must start it by ccl !



---- Additional Comments From drdrulm@yahoo.com 2008-06-12 15:31:33 ----

: is nothing, try searching on:

cats \

(with that slash)

I get at:
http://smfpl.dev.kohalibrary.com/cgi-bin/koha/opac-search.pl?q=cats\
_____________________________________________________________________

Koha error

The following fatal error has occurred:

Trailing \ in regex m/cats\/ at /home/liblime/kohaclone/C4/Search.pm line 1319.

Apache	Server version: Apache/2.2.3 Server built: Jan 31 2008 08:42:27
Koha	3.00.00.087
Koha DB	3.0000087
MySQL	mysql Ver 14.12 Distrib 5.0.32, for pc-linux-gnu (x86_64) using readline 5.2
OS	Linux demo01.ec2.liblime.com 2.6.16.33-xenU #2 SMP Wed Aug 15 17:27:36 SAST 2007 x86_64 GNU/Linux
Perl	5.008008



---- Additional Comments From joe.atzberger@liblime.com 2008-07-07 11:28:52 ----

Darrell --

Works for me (no crash, valid results) on Koha 3.00.00.096.  

--joe



---- Additional Comments From jmf@liblime.com 2008-07-07 21:26:51 ----

Fixing this properly will require a refactor of the search API, and ultimately a query parser. It might be best accomplished by 1. integrating SRU and CQL support into Index Data's Pazpar2 tool, and then using that as the front-end search API for Koha 3.2. I'm changing the Version to 3.2, and keeping it open as a blocker.



---- Additional Comments From frederic@tamil.fr 2009-03-31 08:24:10 ----

Any progress in search refactoring direction? 

Now, a CCL/PQF search is possible, BUT on the first result page, at the bottom, the next-previous buttons don't work at all. 



---- Additional Comments From laurence.lefaucheur@biblibre.com 2009-06-19 13:26:01 ----

Problem searching terms with ()

Test : 
searching cd in title : 96 results
searching (cd) in title : 8777 results, and in this case it is not possible to go to other results pages than the first.
The link from page number is : /koha/catalogue/search.pl?&offset=120&sort_by=relevance, and return to advanced search page



---- Additional Comments From chris@bigballofwax.co.nz 2009-12-12 08:09:56 ----

\ works ok now

() is still a problem, its caused by

if ( $query =~ /(\(|\))/ ) {                                                                                                              
        return (                                                                                                                              
            undef,              $query, $simple_query, $query_cgi,                                                                            
            $query,             $limit, $limit_cgi,    $limit_desc,                                                                           
            $stopwords_removed, 'ccl'                                                                                                         
        );                                                                                                                                    
    }    

Which causes us to bail out before $query_cgi has anything assigned to it, and only the first operand used as the search.
I think that not trying to handle nested queries is a valid stance for 3.2.0 
That would mean that you can search on phrases containing ().

I have sent a patch for that, and will send a patch fixing the : error too.

Search.pm is slated for a total rewrite for 3.4.0, but I think that having it working (and losing the nested queries) is ok for 3.2.0 .. galen?



---- Additional Comments From chris@bigballofwax.co.nz 2009-12-12 08:17:27 ----

Created an attachment
Patch to fix searching on ()





---- Additional Comments From chris@bigballofwax.co.nz 2009-12-12 09:08:28 ----

Created an attachment
Patch to stop : blocking searches





---- Additional Comments From nengard@gmail.com 2009-12-13 22:06:05 ----

*** http://bugs.koha.org/cgi-bin/bugzilla3/show_bug.cgi?id=2702 has been marked as a duplicate of this bug. ***



---- Additional Comments From colin.campbell@ptfs-europe.com 2010-01-28 14:36:34 ----

The second patch seems to have the unwelcome side effect of of breaking search links in opac and staff e.g. when opac-search.pl id called with q=au:Name the au:Name search fails



---- Additional Comments From colin.campbell@ptfs-europe.com 2010-02-02 09:55:07 ----

Chris Nighswonger has posted a patch referring to http://bugs.koha.org/cgi-bin/bugzilla3/show_bug.cgi?id=4074 to address the side effect reported in the second patch.



---- Additional Comments From chris@bigballofwax.co.nz 2010-02-11 20:20:30 ----

This has been patched to a working extent for the 3.2 release

C4/Search.pm needs a total rewrite and is on the cards for 3.4 so shifting this to that.



--- Bug imported by chris@bigballofwax.co.nz 2010-05-21 00:34 UTC  ---

This bug was previously known as _bug_ 1807 at http://bugs.koha.org/cgi-bin/bugzilla3/show_bug.cgi?id=1807
Imported an attachment (id=370)
Imported an attachment (id=371)

Actual time not defined. Setting to 0.0
CC member jwagner@ptfs.com does not have an account here
CC member laurence.lefaucheur@biblibre.com does not have an account here
The original submitter of attachment 370 [details] [review] is unknown.
   Reassigning to the person who moved it here: chris@bigballofwax.co.nz.
The original submitter of attachment 371 [details] [review] is unknown.
   Reassigning to the person who moved it here: chris@bigballofwax.co.nz.

Comment 1 Sharon Moreland 2010-07-13 20:13:20 UTC
Question mark doesn't work either.  Example, searching for the movie "Are we there yet?" retrieves no results, but the same search without the question mark "?" does.
Comment 2 Chris Cormack 2010-12-13 01:19:52 UTC
Needs to be fixed properly for 3.4
Comment 3 Nicole C. Engard 2011-02-28 19:18:01 UTC
I'm getting no results if a subject heading has parens ( ) in it.  


/cgi-bin/koha/opac-search.pl?q=su:XHTML%20%28Document%20markup%20language%29
Comment 4 Magnus Enger 2011-05-16 10:17:19 UTC
Tested with simple search in the OPAC, on current master. Looks like : has been fixed, but not (). 

I created a record with 
245$a = Code
245$b = and other laws (of cyberspace) 
100$a = Lessig, Lawrence

* COLON

These searches work: 

code :
code : lessig
code : and other
code : and cyberspace

while this does not work: 

code : and

but that might be because "and" is seen as a boolean operator ans should have something after it? 

* PARENTHESIS

This works: 

cyberspace
(of cyberspace) 
Code and other laws
Code : and other laws

but, alas, not this: 

Code and other laws (of cyberspace) 
Code : and other laws (of cyberspace) 

Marking as "PATCH-Sent" and "Failed QA" to make sure it shows up in searches.
Comment 5 Chris Nighswonger 2011-09-07 18:15:34 UTC
Bumping this to master based on the last comment.
Comment 6 Paul Poulain 2011-10-26 14:36:35 UTC
() still not working in 3.6.0 release. the : behaviour is fixed, as well as the ? (comment 1)
Comment 7 MJ Ray (software.coop) 2012-06-10 12:18:43 UTC
I have tested this on master with a search like
   Code and other laws (of cyberspace)
and it does not work when it should.

I have checked that the attached patches are in git, so I am marking them obsolete and putting this bug back to ASSIGNED which I think is more accurate and nearer the start of the lifecycle than Failed QA.
Comment 8 Jesse Maseto 2014-10-01 19:56:09 UTC

//Fix subject search by striping parenthesis from url links
$('#catalogue_detail_biblio a[href*="su:"]').attr('href', function(_,v){
return v.replace(/(\(|\))/g,'')
});

This works for both the staff search and OPAC.
Comment 9 Christopher Brannon 2014-10-23 14:43:15 UTC
(In reply to Jesse Maseto from comment #8)
> 
> //Fix subject search by striping parenthesis from url links
> $('#catalogue_detail_biblio a[href*="su:"]').attr('href', function(_,v){
> return v.replace(/(\(|\))/g,'')
> });
> 
> This works for both the staff search and OPAC.

Actually had to update this jquery to catch more.  Simply striping the parenthesis isn't enough, because sometimes it will mash words together.  And sometimes replacing it with a space throws off the search as well.  This modification seems to work better at fixing the issue:

//Fix subject search by striping parenthesis from url links
$('#catalogue_detail_biblio a[href*="su:"]').attr('href', function(_,v){
    return v.replace(/(\w)(\(|\))(\w)/g,'$1 $3')
}).attr('href', function(_,v){
    return v.replace(/(\(|\))/g,'')
});
//End fix subject search

I don't know if it addresses all situations, but I've been tweaking it as I come across new situations.

Christopher
Comment 10 John Andrews 2015-01-13 23:26:10 UTC
The JS solution is working brilliantly in the OPAC, but there is no change on the staff side when I include this in intranetuserjs -- any suggestions on how to make it work there as well?
Comment 11 Christopher Brannon 2015-01-14 16:36:13 UTC
(In reply to John Andrews from comment #10)
> The JS solution is working brilliantly in the OPAC, but there is no change
> on the staff side when I include this in intranetuserjs -- any suggestions
> on how to make it work there as well?

It's working on our end.  Perhaps there is a conflicting jQuery?  What version of Koha on you on?

Christopher
Comment 12 John Andrews 2015-01-14 16:47:00 UTC
We're on 3.16.03.000. 

There are no other scripts in intranetuserjs that should affect details pages at all so I'm not sure where to even begin looking for a conflict.
Comment 13 Christopher Brannon 2015-01-14 16:49:21 UTC
(In reply to John Andrews from comment #12)
> We're on 3.16.03.000. 
> 
> There are no other scripts in intranetuserjs that should affect details
> pages at all so I'm not sure where to even begin looking for a conflict.

There may be conditions it is not able to catch.  Have you tried other searches?

Christopher
Comment 14 John Andrews 2015-01-14 17:21:26 UTC
Have tried various searches -- links with no parentheses work fine, all links I've found with parentheses return the broken search results. Cache cleared, and double-checked to make sure I copied the full script -- everything *looks* like it should be working correctly, but we still only get parentheses removed in the opac and not the staff client.

John
Comment 15 Christopher Brannon 2015-01-14 17:35:37 UTC
Sorry.  I wish I could figure out what's stopping it from working.

I guess someone will actually need to fix this.  :/

Christopher
Comment 16 Nicole C. Engard 2015-01-29 20:21:41 UTC Comment hidden (obsolete)
Comment 17 Nicole C. Engard 2015-01-29 20:22:47 UTC
This patch was written by Winona Salesky - I'm hoping that came across in the patch.

Nicole
Comment 18 Nick Clemens 2015-01-31 16:32:32 UTC Comment hidden (obsolete)
Comment 19 Nick Clemens 2015-01-31 16:37:31 UTC
As far as I could see, this bug only manifested when using ICU Zebra indexing.  This isn't default so wasn't on my VM and I lost facets when turning on so a second sign off wouldn't be out of line.

Other than that the patch did correct the issue and worked fine in both staff and opac
Comment 20 Katrin Fischer 2015-02-01 22:17:06 UTC
Comment on attachment 35646 [details] [review]
[SIGNED OFF] Bug 1807: Remove parens from links to fix searches

Hi Winona,

don't worry, your patch hasn't gone missing! I moved it to bug 13650, as it deals with the subject links perfectly, but didn't fix the general problem that entering search terms with () is a problem. I'd like to keep this bug open in hope that gets fixed too at some point.
Comment 21 Patrick Robitaille 2018-04-20 14:05:59 UTC
Hi, 

This bug still persists in version 17.05 of Koha.Does anyone work on this problem?

When you have a collection title with parenthesis, the search does not work. 

Thanks, Patrick.
Comment 22 Jonathan Druart 2019-12-12 09:37:00 UTC
Seems to be working fine with elastic.
Comment 23 David Cook 2021-01-05 00:32:35 UTC
I can't reproduce this problem in Zebra 2.0.59 when using ICU using the following queries:

http://localhost:8081/cgi-bin/koha/catalogue/search.pl?q=cd
http://localhost:8081/cgi-bin/koha/catalogue/search.pl?q=%28cd%29

If someone can provide me with the steps to reproduce the problem, I could look into this more.