Bug 8610

Summary:	Normalize records imported from Z39.50 servers to prevent Zebra search problems.
Product:	Koha	Reporter:	Frédérick Capovilla <fcapovilla>
Component:	Z39.50 / SRU / OpenSearch Servers	Assignee:	Frédérick Capovilla <fcapovilla>
Status:	CLOSED FIXED	QA Contact:
Severity:	normal
Priority:	P5 - low	CC:	chris, jwagner, m.de.rooy, paul.poulain
Version:	3.10
Hardware:	All
OS:	All
Change sponsored?:	Sponsored	Patch complexity:	---
Documentation contact:		Documentation submission:
Text to go in the release notes:		Version(s) released in:
Circulation function:
Attachments:	Normalize records imported from Z39.50 servers. Signed patch

Description Frédérick Capovilla 2012-08-09 17:44:35 UTC Comment hidden (obsolete)

Created attachment 11512 [details] [review]
Normalize records imported from Z39.50 servers.

Records with separated diacritics might not show up in Zebra searches if
the search terms use accented characters, because typed UTF-8 accented characters normally use combined diacritics.

Some Z39.50 server may use the MARC-8 encoding, which uses separated
diacritics. By forcing a normalization, all imported records will have
combined diacritics.

Example serveur using separated diacritics (MARC-21): http://www.collectionscanada.gc.ca/amicus/006002-400-f.html

Hostname: amicus.collectionscanada.gc.ca
Port: 210
Database: BN
Syntax: MARC21/USMARC
Encoding: MARC-8

Comment 1 Frédérick Capovilla 2012-08-09 17:45:05 UTC

Patch sponsored by the CCSR ( http://www.ccsr.qc.ca )

Comment 2 Marcel de Rooy 2012-10-08 12:13:16 UTC

Works fine.
My only point would be that function MarcToUTF8Record should have returned a normalized record already. Now we receive it and need to call SetUTF8Flag again (from the same module!).
Signing off.

Comment 3 Marcel de Rooy 2012-10-08 12:56:46 UTC

QA Comment:
Code looks good. See also previous comment.
A followup could handle the output of MarcToUTF8Record in the scripts bulkmarcimport.pl, bulkauthimport.pl and svc/new_bib. In those three cases a similar correction could be necessary.

Comment 4 Marcel de Rooy 2012-10-08 13:16:08 UTC

Created attachment 12732 [details] [review]
Signed patch

Comment 5 Paul Poulain 2012-10-08 16:47:39 UTC

Patch pushed to master

Comment 6 Chris Cormack 2013-04-21 00:34:23 UTC

Released in 3.10.0