Bug 8065

Summary: chinese word search result all chinese biblios record
Product: Koha Reporter: Thomas Tsai <tlinux.tsai>
Component: SearchingAssignee: Galen Charlton <gmcharlt>
Status: CLOSED INVALID QA Contact:
Severity: major    
Priority: P5 - low CC: katrin.fischer
Version: 3.6   
Hardware: PC   
OS: Linux   
Change sponsored?: --- Patch complexity: ---
Documentation contact: Documentation submission:
Text to go in the release notes:
Version(s) released in:

Description Thomas Tsai 2012-05-10 09:09:46 UTC
search any chinese charset, return all chinese records. I can get the correct record and all other record even not match my search term.

yaz-client and idrebra version is
yaz: 4.2.18-1build1 and try to upgrade 4.2.32, get same error result.
idzebra-2.0: 2.0.44-3

I also use yaz-cline to do search test, the result log here:

yaz-client unix:/var/run/koha/zebradb/bibliosocket
Connecting...OK.
Sent initrequest.
Connection accepted by v3 target.
ID     : 81
Name   : Zebra Information Server/GFS/YAZ
Version: 4.2.32 cc1a58f855146ca26d117f6424284beda0e7382b
Options: search present delSet triggerResourceCtrl scan sort extendedServices namedResultSets
Elapsed: 0.004470
Z> authentication koha/test
Authentication set to Open (koha/ilovekoha)
Z> base biblios
Z> querycharset UTF-8

              //correct result in search english pattern //

Z> f @attr 1=1016 @attr 4=6 @attr 5=1  Andai
Sent searchRequest.
Received SearchResponse.
Search was a success.
Number of hits: 1, setno 1
SearchResult-1: term=Andai cnt=1
records returned: 0
Elapsed: 0.033256
Z> s
Sent presentRequest (1+1).
Records: 1
[biblios]Record type: USmarc
00941nam a22002777a 4500
001 15867806
003 OSt22
005 20120509001412.0
008 090812s2009    io a     b    000 0 ind
010    $a   2009315608
020    $a 9789792121384
040    $a DLC $c DLC
042    $a lcode
050 00 $a MLCSE 2009/00577 (J)
245 00 $a Andai presiden kita sehebat Harry Potter : $b kumpulan esai orang muda Indonesia tentang presiden / $c Agenda 18.
....


           // incorrect search result //
Z>f @attr 1=1016 @attr 4=6 @attr 5=1  孫
Sent searchRequest.
Received SearchResponse.
Search was a success.

Number of hits: 2, setno 2  <------ only one record is correct, but i get all records here

SearchResult-1: term=孫 cnt=2   
records returned: 0
Elapsed: 0.002453
Z> find kw,wrdl,rtrn=往
Sent searchRequest.
Received SearchResponse.
Search was a success.
Number of hits: 2, setno 3
SearchResult-1: term=往 cnt=2   
records returned: 0
Elapsed: 0.002350
Z> s
Sent presentRequest (1+1).
Records: 1                 <---- correct one
[biblios]Record type: USmarc
00741cam a2200253 i 4500
001 ptl43000661
003 OSt
005 20120326193737.0
008 050401s2005    ch a          000 0 chi d
020    $a 9577457568 (精裝) : $c 新臺幣250元
040    $a ISBN $b chi $d PTL $c ISBN $d PTTCL
041 0  $a chi
090    $a 859.6 $b 1262-3
100 1  $a 孫晴峰.                 <---- match pattern here
245 10 $a 獅子燙頭髮 /  $c 孫晴峰文 ; 龐雅文圖.
250    $a 初版.
260    $a 臺北市 : $b 格林文化, $c 民94[2005]
300    $a 1 冊 : $b 彩圖 ; $c 30 公分.
440  0 $a 格林名家繪本館.
500    $a 國語注音.
700 1  $a 龐雅文.
942    $2 ddc $c BK
999    $c 2 $d 2
952    $w 2012-03-26 $7 0 $r 2012-03-26 $4 0 $2 ddc $0 0 $9 1 $b ccl $y BK $1 0 $d 2012-03-26 $a ccl

nextResultSetPosition = 2
Elapsed: 0.000898
Z>
  // this is incorrect record, no any pattern match my search term.
Sent presentRequest (2+1).
Records: 1
[biblios]Record type: USmarc
00305nam a22000857a 4500
008 120409t        xxu||||| |||| 00| 0 eng d
100    $c 蔡育欽 $l 中文
245    $a 中文書本測試專用-國網中心
999    $c 3 $d 3
952    $w 2012-04-09 $r 2012-04-09 $4 0 $0 0 $6 564738_000000000000000 $9 2 $b ccl $1 0 $o 564738 $d 2012-04-09 $7 0 $2 ddc $y BK $a ccl

nextResultSetPosition = 3
Comment 1 Katrin Fischer 2012-05-10 10:59:33 UTC
Hi Thomas, 
this is not a bug but a configuration issue. In order to make it work you need to activate ICU indexing. It's available as an option at installation with Koha 3.8. It can be used with older versions too:
http://lists.katipo.co.nz/public/koha/2012-January/031714.html
Comment 2 Thomas Tsai 2012-05-11 01:45:49 UTC
Hi,

got it, thank you. 


I follow the post 
http://lists.katipo.co.nz/public/koha/2011-January/027171.html

and now i can get good search result.

Thank You!

BR, Thomas.