Bug 18910

Summary: Regression: Z39.50 wrong conversion in Unimarc by Bug 18152
Product: Koha Reporter: Zeno Tajoli <ztajoli>
Component: Z39.50 / SRU / OpenSearch ServersAssignee: Jonathan Druart <jonathan.druart>
Status: CLOSED FIXED QA Contact: Testopia <testopia>
Severity: critical    
Priority: P5 - low CC: acovaliov, baptiste.wojtkowski, fridolin.somers, jonathan.druart, katrin.fischer, m.de.rooy, mtj
Version: master   
Hardware: All   
OS: All   
See Also: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18935
Change sponsored?: --- Patch complexity: Small patch
Who signed the patch off: Text to go in the release notes:
Version(s) released in:
Bug Depends on: 18152    
Bug Blocks: 18935    
Attachments: Bug 18910: Revert "Bug 18152: Add tests"
Bug 18910: Revert "Bug 18152 : fix unimarc label in SetMarcUnicodeFlag"
Bug 18910: Revert "Bug 18152: Add tests"
Bug 18910: Revert "Bug 18152 : fix unimarc label in SetMarcUnicodeFlag"

Description Zeno Tajoli 2017-07-07 13:59:29 UTC
Bug 18152 created a regression on z39.50 derivate cataloguing in Unimarc.
Now if I derivate from a z39.50 with source date in  ISO5426, data are imported well in table "import_record".
But if I try to see the with ../catalogue/showmarc.pl or import them into cataloguing form with ../cataloguing/z3950_search.pl#, date are converted in a wrong way.

To see the bug you need:
-- a Unimarc setup
-- as languge 'en' is ok
-- a z39.50 server with ISO5426 encoding, for example:
Hostname: catalogo.bnportugal.pt
Port: 210
Db: bn
Syntax: UNIMARC
Encoding: ISO_5426

In fact you can cancel this problem on z39.50 with this diff:
diff --git a/Charset.pm_old b/Charset.pm
index 50d8266..9212aeb 100644
--- a/Charset.pm_old
+++ b/Charset.pm
@@ -324,6 +324,7 @@ sub SetMarcUnicodeFlag {
     my $marc_record = shift;
     my $marc_flavour = shift; # || C4::Context->preference("marcflavour");

+    $marc_record->encoding('UTF-8');
     if ($marc_flavour eq 'MARC21' || $marc_flavour eq 'NORMARC') {
         my $leader = $marc_record->leader();
         substr($leader, 9, 1) = 'a';

But I think the bug is more complex, probably a revert is not enough.
Comment 1 Zeno Tajoli 2017-07-07 14:06:41 UTC
This bug is not easy to find.
Here the mail thread about it:
http://koha.1045719.n5.nabble.com/Z39-50-encoding-problem-tested-on-2-koha-servers-td5938540.html

The bug is present on 16.11, 17.05 and master
Comment 2 Jonathan Druart 2017-07-17 19:01:42 UTC
Created attachment 65083 [details] [review]
Bug 18910: Revert "Bug 18152: Add tests"

This reverts commit a807d7199ea025aa507e0ca7e3ff2330902844f8.
Comment 3 Jonathan Druart 2017-07-17 19:04:01 UTC
Created attachment 65084 [details] [review]
Bug 18910: Revert "Bug 18152 : fix unimarc label in SetMarcUnicodeFlag"

This reverts commit bf551a07222a7b9beffd7ca6cba38c675aa3ab28.
Comment 4 Zeno Tajoli 2017-07-17 21:02:44 UTC
Jonathan, OK on revert, but I sugggest also a comment about bad Unimard haldling in MARC::Record library.
Comment 5 Fridolin SOMERS 2017-07-19 14:18:18 UTC
Created attachment 65102 [details] [review]
Bug 18910: Revert "Bug 18152: Add tests"

This reverts commit a807d7199ea025aa507e0ca7e3ff2330902844f8.

Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
Comment 6 Fridolin SOMERS 2017-07-19 14:18:36 UTC
Created attachment 65103 [details] [review]
Bug 18910: Revert "Bug 18152 : fix unimarc label in SetMarcUnicodeFlag"

This reverts commit bf551a07222a7b9beffd7ca6cba38c675aa3ab28.

Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
Comment 7 Jonathan Druart 2017-07-26 17:03:25 UTC
Skipping QA, we need it in the next stable releases.
Comment 8 Jonathan Druart 2017-07-26 17:03:42 UTC
Patches pushed to master for 17.11
Comment 9 Mason James 2017-07-27 09:05:13 UTC
I'll push to 16.05.x branch
Comment 10 Fridolin SOMERS 2017-07-27 09:56:27 UTC
Pushed to 17.05.x, will be in 17.05.02.

Tested on UNIMARC db with BNF z3950 like in our demo : https://intranet-demo.biblibre.com
Comment 11 Katrin Fischer 2017-07-27 23:16:28 UTC
These patches have been pushed to 16.11.x and will be in 16.11.10.
Comment 12 Mason James 2017-07-31 11:02:21 UTC
Pushed to 16.05.x, for 16.05.15 release