8610 – Normalize records imported from Z39.50 servers to prevent Zebra search problems.

Bug 8610 - Normalize records imported from Z39.50 servers to prevent Zebra search problems.

Summary: Normalize records imported from Z39.50 servers to prevent Zebra search problems.

Status:	CLOSED FIXED

Alias:	None

Product:	Koha
Classification:	Unclassified
Component:	Z39.50 / SRU / OpenSearch Servers (show other bugs)
Version:	3.10
Hardware:	All All

Importance:	P5 - low normal
Assignee:	Frédérick Capovilla
QA Contact:

URL:
Keywords:

Depends on:
Blocks:

Reported:	2012-08-09 17:44 UTC by Frédérick Capovilla
Modified:	2013-12-05 20:05 UTC (History)
CC List:	4 users (show)

See Also:
Change sponsored?:	Sponsored
Patch complexity:	---
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:
Circulation function:

Attachments
Normalize records imported from Z39.50 servers. (3.42 KB, patch) 2012-08-09 17:44 UTC, Frédérick Capovilla	Details \| Diff \| Splinter Review
Signed patch (3.53 KB, patch) 2012-10-08 13:16 UTC, Marcel de Rooy	Details \| Diff \| Splinter Review
Show Obsolete (1) View All Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.

Description Frédérick Capovilla 2012-08-09 17:44:35 UTC Comment hidden (obsolete)

Created attachment 11512 [details] [review]
Normalize records imported from Z39.50 servers.

Records with separated diacritics might not show up in Zebra searches if
the search terms use accented characters, because typed UTF-8 accented characters normally use combined diacritics.

Some Z39.50 server may use the MARC-8 encoding, which uses separated
diacritics. By forcing a normalization, all imported records will have
combined diacritics.

Example serveur using separated diacritics (MARC-21): http://www.collectionscanada.gc.ca/amicus/006002-400-f.html

Hostname: amicus.collectionscanada.gc.ca
Port: 210
Database: BN
Syntax: MARC21/USMARC
Encoding: MARC-8

Comment 1 Frédérick Capovilla 2012-08-09 17:45:05 UTC

Patch sponsored by the CCSR ( http://www.ccsr.qc.ca )

Comment 2 Marcel de Rooy 2012-10-08 12:13:16 UTC

Works fine.
My only point would be that function MarcToUTF8Record should have returned a normalized record already. Now we receive it and need to call SetUTF8Flag again (from the same module!).
Signing off.

Comment 3 Marcel de Rooy 2012-10-08 12:56:46 UTC

QA Comment:
Code looks good. See also previous comment.
A followup could handle the output of MarcToUTF8Record in the scripts bulkmarcimport.pl, bulkauthimport.pl and svc/new_bib. In those three cases a similar correction could be necessary.

Comment 4 Marcel de Rooy 2012-10-08 13:16:08 UTC

Created attachment 12732 [details] [review]
Signed patch

Comment 5 Paul Poulain 2012-10-08 16:47:39 UTC

Patch pushed to master

Comment 6 Chris Cormack 2013-04-21 00:34:23 UTC

Released in 3.10.0