Bug 14431

Summary: Encoding issues - search in staff
Product: Koha Reporter: Jonathan Druart <jonathan.druart>
Component: SearchingAssignee: Jonathan Druart <jonathan.druart>
Status: CLOSED FIXED QA Contact: Testopia <testopia>
Severity: major    
Priority: P5 - low CC: arouss1980, chris, kyle, mtompset, sergeinik, suzonduislm52, tomascohen
Version: Main   
Hardware: All   
OS: All   
Change sponsored?: --- Patch complexity: ---
Documentation contact: Documentation submission:
Text to go in the release notes:
Version(s) released in:
Bug Depends on: 11944    
Bug Blocks:    
Attachments: Bug 14431: FIX encoding issues in search (staff client)
Bug 14431: FIX encoding issues in search (staff client)
[PASSED QA] Bug 14431: FIX encoding issues in search (staff client)

Description Jonathan Druart 2015-06-22 10:44:31 UTC
It certainly depends on the Encode version.

To reproduce:

1) /cgi-bin/koha/catalogue/search.pl?q=ééé&op=Submit

" No results match your search for 'kw,wrdl: ���' in my library Catalog. "

2) /cgi-bin/koha/catalogue/search.pl?q=ກ

Cannot decode string with wide characters at /usr/lib/i386-linux-gnu/perl/5.20/Encode.pm line 215.

My encode version is 2.60 (`pmvers Encode`).
Comment 1 Chris Cormack 2015-06-22 10:46:24 UTC
Encode version 2.44 (what is in Debian wheezy) is fine.
Comment 2 Jonathan Druart 2015-06-22 10:55:41 UTC Comment hidden (obsolete)
Comment 3 Mark Tompsett 2015-06-23 03:10:52 UTC
(In reply to Jonathan Druart from comment #2)
> 1) /cgi-bin/koha/catalogue/search.pl?q=ééé&op=Submit
> You should get
> " No results match your search for 'kw,wrdl: ���' in my library Catalog."
> 2) /cgi-bin/koha/catalogue/search.pl?q=ກ
> You should get
> Cannot decode string with wide characters at
> /usr/lib/i386-linux-gnu/perl/5.20/Encode.pm line 215.

I didn't know about pmvers before, but a sudo apt-get install pmtools solved that.

Actually, before the patch, I got similar to this on:
- Ubuntu 14.04, Encode 2.57, perl 5.18.2
- Debian 8, Encode 2.63, perl 5.20.2

Before the patch on:
- Debian 7, Encode 2.42_01, perl 5.14.2
There were no apparent problems.

Given that Chris said it was 2.44, I upgraded my Encode to 2.44 (sudo apt-get install libencode-perl) and retested.

After the patch, problems were corrected as expected, and no apparent regressions.

Is this sufficient for a sign off? I will sign off, assuming this is. :)
Comment 4 Mark Tompsett 2015-06-23 03:13:46 UTC Comment hidden (obsolete)
Comment 5 Jonathan Druart 2015-06-23 08:31:01 UTC
*** Bug 14286 has been marked as a duplicate of this bug. ***
Comment 6 Andreas Roussos 2015-06-26 16:18:28 UTC
I can confirm that applying the patch fixes the issue for me (searching for Greek characters was broken in the staff interface). FWIW I'm using koha 3.20 on Ubuntu 15.04 and my Encode.pm is at version 2.60.
Comment 7 Kyle M Hall 2015-06-26 18:37:52 UTC
Created attachment 40693 [details] [review]
[PASSED QA] Bug 14431: FIX encoding issues in search (staff client)
Comment 8 Tomás Cohen Arazi 2015-06-26 19:33:52 UTC
Please add a regression test in t/db_dependent/www/search_utf8.t
Comment 9 Jonathan Druart 2015-07-02 09:51:55 UTC
(In reply to Tomás Cohen Arazi from comment #8)
> Please add a regression test in t/db_dependent/www/search_utf8.t

I'd say the existing tests are enough. Could someone confirm?

Tomas, looking at http://jenkins.koha-community.org/job/Koha_Master_D7/lastBuild/testReport/%28root%29/ the tests are not launch by jenkins, they should.
Comment 10 Tomás Cohen Arazi 2015-07-02 10:40:28 UTC
(In reply to Jonathan Druart from comment #9)
> (In reply to Tomás Cohen Arazi from comment #8)
> > Please add a regression test in t/db_dependent/www/search_utf8.t
> 
> I'd say the existing tests are enough. Could someone confirm?
> 
> Tomas, looking at
> http://jenkins.koha-community.org/job/Koha_Master_D7/lastBuild/testReport/
> %28root%29/ the tests are not launch by jenkins, they should.

You are right, I only set those tests to be run on 14.04 when we pushed them, and left the other VMs 'for later'. I'll try to set it. Sorry for that.
Comment 11 Tomás Cohen Arazi 2015-07-07 18:29:06 UTC
Patch pushed to master.

Thanks Jonathan!
Comment 12 Chris Cormack 2015-07-13 23:42:47 UTC
Pushed to 3.20.x will be in 3.20.2
Comment 13 Indranil Das Gupta 2015-07-18 21:25:33 UTC
*** Bug 14561 has been marked as a duplicate of this bug. ***