Bug 9522 - Koha returns &lt, &gt, etc in the record instead of <, >, etc when fulfilling a Z39.50 request
Summary: Koha returns &lt, &gt, etc in the record instead of <, >, etc when fulfilling...
Status: CLOSED FIXED
Alias: None
Product: Koha
Classification: Unclassified
Component: Z39.50 / SRU / OpenSearch Servers (show other bugs)
Version: 3.10
Hardware: All All
: P5 - low normal (vote)
Assignee: Galen Charlton
QA Contact:
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2013-01-31 19:57 UTC by Joseph Alway
Modified: 2015-12-03 22:00 UTC (History)
5 users (show)

See Also:
Change sponsored?: ---
Patch complexity: ---
Text to go in the release notes:
Version(s) released in:


Attachments
Test Record - Linux Journal (2.18 KB, application/octet-stream)
2013-01-31 20:09 UTC, Joseph Alway
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Joseph Alway 2013-01-31 19:57:51 UTC
When searching Koha using a Z39.50 client you get &lt, &gt, etc returned in the record instead of <, >, etc.

I have installed a fresh version of debian and koha 3.10.2. I have imported a "fresh" record using the Z39.50 search. I grabbed the "Linux Journal" record from the LC Z39.50 server.


I have connected to Koha's Z39.50 server using the following command: yaz-client -c /etc/koha/zebradb/ccl.properties localhost:9999/biblios

I run the command find linux and then show. I get a record that returns &lt and &gt, etc. instead of <, >, etc.

This also happens when I connect to Koha's Z39.50 server using TLC's Cataloging module. So, multiple Z39.50 clients having the same issue with Koha's Z39.50 server giving them &lt xml entities instead of the correct character <, etc.
Comment 1 Jane Wagner 2013-01-31 20:08:27 UTC
I ran into this problem recently and we did various tests.  The MARC records within Koha were clean; extracting them via Tools, Bib Export also produced clean records.  Extracting them via a z39.50 connection to the Koha database produced records with the funky characters.  We noticed most often the substitution of &amp; for an ampersand.  I also tested against other z39.50 targets.  The problem seems to be limited to Koha's z39.50 export.

We found that the problem disappeared when we tested against a site running Solr instead of zebra indexing, so it is further specific to zebra in some manner.
Comment 2 Joseph Alway 2013-01-31 20:09:20 UTC
Created attachment 14986 [details]
Test Record - Linux Journal

In theory any record with the <, >, etc characters should give the same problem.
Comment 3 Joseph Alway 2013-01-31 20:24:09 UTC
(In reply to comment #1)
> I ran into this problem recently and we did various tests.  The MARC records
> within Koha were clean; extracting them via Tools, Bib Export also produced
> clean records.  Extracting them via a z39.50 connection to the Koha database
> produced records with the funky characters.  We noticed most often the
> substitution of &amp; for an ampersand.  I also tested against other z39.50
> targets.  The problem seems to be limited to Koha's z39.50 export.
> 
> We found that the problem disappeared when we tested against a site running
> Solr instead of zebra indexing, so it is further specific to zebra in some
> manner.

The Linux Journal test record I attached was exported from the Koha Server I am having this issue with. I am having pretty much all of those same symptoms. I have not tried using Solr.
Comment 4 Gaetan Boisson 2013-02-13 11:30:47 UTC
It seems this is linked to a zebra problem indeed, see bug 3326 for more info. This also affects result pages when they are displayed via XSLT.
Comment 5 Joseph Alway 2013-02-13 19:11:22 UTC
I ran the command "sudo koha-rebuild-zebra -f kohatest" in a test machine and that seemed to fix the problem. When I ran the command for my production machine it did not do anything. They both should be setup the same.

The only thing I did different in my test machine is that I commented out the opt_xml lines in the file /usr/sbin/koha-rebuild-zebra. That didn't seem to do anything, so I put it back the way it was originally by removing the comment.
Comment 6 Fridolin Somers 2013-10-23 15:02:54 UTC
I confirm this bug.

It appears when Zebra database is in GRS1 and when rebuild uses XML (rebuild -x) and retrieving in MARC (iso2709).

See :
http://search.cpan.org/~mirk/Net-Z3950-ZOOM/lib/ZOOM.pod#raw%28%29

Best solution is to avoid GRS-1.

But someone must confirm it works using DOM.
Comment 7 Jonathan Druart 2014-12-11 15:27:18 UTC
(In reply to Fridolin SOMERS from comment #6)
> But someone must confirm it works using DOM.

I confirm it works.

mysql> select title from biblio where biblionumber=42;
=> Une Fête monstre <test>

% yaz-client localhost:9998/biblios
Connecting...OK.
Sent initrequest.
Connection accepted by v3 target.
ID     : 81
Name   : Zebra Information Server/GFS/YAZ
Version: 4.2.30 98864b44c654645bc16b2c54f822dc2e45a93031
Options: search present delSet triggerResourceCtrl scan sort extendedServices namedResultSets
Elapsed: 0.001121
Z> format unimarc
Z> f "Une fete monstre"
Sent searchRequest.
Received SearchResponse.
Search was a success.
Number of hits: 3, setno 1
SearchResult-1: term=Une cnt=760, term=fete cnt=29, term=monstre cnt=13
records returned: 0
Elapsed: 0.035910
Z> show
Sent presentRequest (3+1).
Records: 1
[biblios]Record type: Unimarc
00595nam a2200181   4500
001 142
010    $a 9879350061855
090    $a 42
100    $a 19960115              frey50        
200 1  $a Une Fête monstre <test> $f Stéphanie Laslett $g ill. Nigel MacMullen $b LIVR
[SKIP]
Comment 8 Jonathan Druart 2014-12-11 15:28:13 UTC
Please reopen if I am wrong.