Bug 18910 - Regression: Z39.50 wrong conversion in Unimarc by Bug 18152
Summary: Regression: Z39.50 wrong conversion in Unimarc by Bug 18152
Status: Pushed to Stable
Alias: None
Product: Koha
Classification: Unclassified
Component: Z39.50 / SRU / OpenSearch Servers (show other bugs)
Version: master
Hardware: All All
: P5 - low critical (vote)
Assignee: Jonathan Druart
QA Contact: Testopia
URL:
Whiteboard:
Keywords:
Depends on: 18152
Blocks: 18935
  Show dependency treegraph
 
Reported: 2017-07-07 13:59 UTC by Zeno Tajoli
Modified: 2017-07-31 11:02 UTC (History)
7 users (show)

See Also:
Change sponsored?: ---
Patch complexity: Small patch
Bot Control: ---
When did the bot last check this:
Who signed the patch off:
Text to go in the release notes:


Attachments
Bug 18910: Revert "Bug 18152: Add tests" (2.12 KB, patch)
2017-07-17 19:01 UTC, Jonathan Druart
Details | Diff | Splinter Review
Bug 18910: Revert "Bug 18152 : fix unimarc label in SetMarcUnicodeFlag" (856 bytes, patch)
2017-07-17 19:04 UTC, Jonathan Druart
Details | Diff | Splinter Review
Bug 18910: Revert "Bug 18152: Add tests" (2.18 KB, patch)
2017-07-19 14:18 UTC, Fridolin SOMERS
Details | Diff | Splinter Review
Bug 18910: Revert "Bug 18152 : fix unimarc label in SetMarcUnicodeFlag" (912 bytes, patch)
2017-07-19 14:18 UTC, Fridolin SOMERS
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Zeno Tajoli 2017-07-07 13:59:29 UTC
Bug 18152 created a regression on z39.50 derivate cataloguing in Unimarc.
Now if I derivate from a z39.50 with source date in  ISO5426, data are imported well in table "import_record".
But if I try to see the with ../catalogue/showmarc.pl or import them into cataloguing form with ../cataloguing/z3950_search.pl#, date are converted in a wrong way.

To see the bug you need:
-- a Unimarc setup
-- as languge 'en' is ok
-- a z39.50 server with ISO5426 encoding, for example:
Hostname: catalogo.bnportugal.pt
Port: 210
Db: bn
Syntax: UNIMARC
Encoding: ISO_5426

In fact you can cancel this problem on z39.50 with this diff:
diff --git a/Charset.pm_old b/Charset.pm
index 50d8266..9212aeb 100644
--- a/Charset.pm_old
+++ b/Charset.pm
@@ -324,6 +324,7 @@ sub SetMarcUnicodeFlag {
     my $marc_record = shift;
     my $marc_flavour = shift; # || C4::Context->preference("marcflavour");

+    $marc_record->encoding('UTF-8');
     if ($marc_flavour eq 'MARC21' || $marc_flavour eq 'NORMARC') {
         my $leader = $marc_record->leader();
         substr($leader, 9, 1) = 'a';

But I think the bug is more complex, probably a revert is not enough.
Comment 1 Zeno Tajoli 2017-07-07 14:06:41 UTC
This bug is not easy to find.
Here the mail thread about it:
http://koha.1045719.n5.nabble.com/Z39-50-encoding-problem-tested-on-2-koha-servers-td5938540.html

The bug is present on 16.11, 17.05 and master
Comment 2 Jonathan Druart 2017-07-17 19:01:42 UTC
Created attachment 65083 [details] [review]
Bug 18910: Revert "Bug 18152: Add tests"

This reverts commit a807d7199ea025aa507e0ca7e3ff2330902844f8.
Comment 3 Jonathan Druart 2017-07-17 19:04:01 UTC
Created attachment 65084 [details] [review]
Bug 18910: Revert "Bug 18152 : fix unimarc label in SetMarcUnicodeFlag"

This reverts commit bf551a07222a7b9beffd7ca6cba38c675aa3ab28.
Comment 4 Zeno Tajoli 2017-07-17 21:02:44 UTC
Jonathan, OK on revert, but I sugggest also a comment about bad Unimard haldling in MARC::Record library.
Comment 5 Fridolin SOMERS 2017-07-19 14:18:18 UTC
Created attachment 65102 [details] [review]
Bug 18910: Revert "Bug 18152: Add tests"

This reverts commit a807d7199ea025aa507e0ca7e3ff2330902844f8.

Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
Comment 6 Fridolin SOMERS 2017-07-19 14:18:36 UTC
Created attachment 65103 [details] [review]
Bug 18910: Revert "Bug 18152 : fix unimarc label in SetMarcUnicodeFlag"

This reverts commit bf551a07222a7b9beffd7ca6cba38c675aa3ab28.

Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
Comment 7 Jonathan Druart 2017-07-26 17:03:25 UTC
Skipping QA, we need it in the next stable releases.
Comment 8 Jonathan Druart 2017-07-26 17:03:42 UTC
Patches pushed to master for 17.11
Comment 9 Mason James 2017-07-27 09:05:13 UTC
I'll push to 16.05.x branch
Comment 10 Fridolin SOMERS 2017-07-27 09:56:27 UTC
Pushed to 17.05.x, will be in 17.05.02.

Tested on UNIMARC db with BNF z3950 like in our demo : https://intranet-demo.biblibre.com
Comment 11 Katrin Fischer 2017-07-27 23:16:28 UTC
These patches have been pushed to 16.11.x and will be in 16.11.10.
Comment 12 Mason James 2017-07-31 11:02:21 UTC
Pushed to 16.05.x, for 16.05.15 release