As of CGI version 2.64 ( which was a long time again ) Perl's CGI module began supporting the use of the semicolon (;) as a key value pair separator in addition to the ampersand (&) which Koha currently uses. This causes an issue where is search string is truncated at the semicolon. Not only can this cause issues with incorrect searches/results, it can cause malformed CCL which causes Koha to return no search results! Test Plan: 1) Create a record with a 490$a series statement where there is a semicolon at the end 2) Enable OPAC Suppression 3) Search for the record and click the "Series:" link 4) You will get no results This is because the search: se,phr:"Report to the Mississippi Legislature ;" is converted to se,phr:"Report to the Mississippi Legislature When the opac suppression is tacked on, we get malformed CCL: (se,phr="Report to the Mississippi Legislature ) not Suppress=1 This is not an issue if we simply search for se,phr:"Report to the Mississippi Legislature ;" because the search form properly encodes the semicolon. The solution appears to be to make sure all URL elements are properly url encoded and converts all ";" to "%3B". I do notice that in koha-tmpl/intranet-tmpl/prog/en/xslt/MARC21slimUtils-MODS31.xsl we have a template url-encode that may prove useful.
Hi Kyle, Using the MODS url-encode works when you transfer it over to slimUtils, is that what you had in mind or something else? Seems like it sorts the problem nicely. Cheers, Liz
Created attachment 67622 [details] [review] Bug 16804 - unescaped semicolons in XSLT values can cause searches to fail This patch adds a url-encode to the MARC21slimUtils, and applies it specifically to a common place where semicolons can appear in searches, the 490 field. To test: * have OPACSuppression on * have a record with a 490 field a la: 490 _a something ; _v 123 * go to the opac and search for this record * click the Series link from the detail Without this patch, there will be no results. With this patch, you will get relevant results (possibly the same record back) There are probably lots of places this could be applied, and also possibly many other solutions.
Hi Liz, I think we had a similar approach on another bug, but when I tested with Japanese text the links were totally broken. Could you test your patch with some original script in 490?
I've used some Japanese signs in 490 for testing: 日本日本日本;;; The link is broken: http://localhost:8080/cgi-bin/koha/opac-search.pl?q=se,phr:%22%3F%3F%3F%3F%3F%3F%3B%3B%3B%22
I think an easier solution would be to just remove ; for now as this is the most common problem case.
Created attachment 70241 [details] [review] Bug 16804 - semicolons break series search in 490 To test: * have OPACSuppression on * have a record with a 490 field a la: 490 _a something ; _v 123 * go to the opac and search for this record * click the Series link from the detail Without this patch, there will be no results. With this patch, you will get relevant results (possibly the same record back)
Hi Katrin, I put a chopPunctuation on the href attribute, it'll chop of whatever ending punctuation people feel inclined to put there. What do you think? Cheers, Liz
well not *any*, specifically these: .:,;/ '
Created attachment 76064 [details] [review] Bug 16804 - semicolons break series search in 490 To test: * have OPACSuppression on * have a record with a 490 field a la: 490 _a something ; _v 123 * go to the opac and search for this record * click the Series link from the detail Without this patch, there will be no results. With this patch, you will get relevant results (possibly the same record back) Signed-off-by: Joy Nelson <joy@bywatersolutions.com>
Why "* have OPACSuppression on"? Would like to test if bug 14716 also resolves the issue - might be the nicer global fix.
Katrin, I did a retest to confirm. This patch will fix if opac suppress if ON or OFF. joy (In reply to Katrin Fischer from comment #10) > Why "* have OPACSuppression on"? > > Would like to test if bug 14716 also resolves the issue - might be the nicer > global fix.
I feel bug 18304 could be the better solution here. I tested it also fixes the issue. Linking both bugs for now using 'see also'.
Can you verify the bug number Katrin? 18304 is an email issue. (In reply to Katrin Fischer from comment #12) > I feel bug 18304 could be the better solution here. I tested it also fixes > the issue. Linking both bugs for now using 'see also'.
Sorry, too late here: bug 14716
That's funny, my original solution was basically that url-encode one.
Not quite, this is using an exslt function that can do more than the self implemented url-encode that was tried before. Especially original script like Hebrew and Chinese. Try adding a subject 日本 (Japan) in 653. It will work with the new solution, but not with the last one.
Moving this to In Discussion. Leaving it in SO is confusing. We seem to wait for progress on 14716.
Marking resolved fixed as the url-encoding with the exslt function now solves this nicely. *** This bug has been marked as a duplicate of bug 14716 ***