Bug 14196 - Encoding error adding new biblio in custom framework
Summary: Encoding error adding new biblio in custom framework
Status: CLOSED WORKSFORME
Alias: None
Product: Koha
Classification: Unclassified
Component: Cataloging (show other bugs)
Version: 3.18
Hardware: All All
: P5 - low normal (vote)
Assignee: Galen Charlton
QA Contact: Testopia
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2015-05-13 14:19 UTC by pablo.jimenez
Modified: 2017-06-14 22:10 UTC (History)
2 users (show)

See Also:
Change sponsored?: ---
Patch complexity: ---
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description pablo.jimenez 2015-05-13 14:19:34 UTC
Hi.

I have created my own framework (from BKS), with less fields in the structure, and with new Labels in spanish.

When I try to add a new record (biblio) with ñ or Ñ character, the h1 field remove those characters, and diacritical marks are showed wrong (Jiménez as Jimňez). 

The info in the database is correct.

This is not happening in detail.tt

OS: CenOS 7
Locale: 
LANG=es_ES.utf8
LC_CTYPE="es_ES.utf8"
LC_NUMERIC="es_ES.utf8"
LC_TIME="es_ES.utf8"
LC_COLLATE="es_ES.utf8"
LC_MONETARY="es_ES.utf8"
LC_MESSAGES="es_ES.utf8"
LC_PAPER="es_ES.utf8"
LC_NAME="es_ES.utf8"
LC_ADDRESS="es_ES.utf8"
LC_TELEPHONE="es_ES.utf8"
LC_MEASUREMENT="es_ES.utf8"
LC_IDENTIFICATION="es_ES.utf8"
LC_ALL=

Apache with AddDefaultCharset UTF-8

Mysql 5.6 char variables:
+--------------------------+----------------------------+
| Variable_name            | Value                      |
+--------------------------+----------------------------+
| character_set_client     | utf8                       |
| character_set_connection | utf8                       |
| character_set_database   | utf8                       |
| character_set_filesystem | binary                     |
| character_set_results    | utf8                       |
| character_set_server     | utf8                       |
| character_set_system     | utf8                       |
| character_sets_dir       | /usr/share/mysql/charsets/ |
+--------------------------+----------------------------+


Regards.
Comment 1 pablo.jimenez 2015-05-13 14:34:31 UTC
I was wrong about the data in database, after a query it looks like is double utf-8 encoded

SELECT title, LENGTH(title), CHAR_LENGTH(title) FROM biblio WHERE LENGTH(title) != CHAR_LENGTH(title);
+----------------------+---------------+--------------------+
| title                | LENGTH(title) | CHAR_LENGTH(title) |
+----------------------+---------------+--------------------+
| España               |             7 |                  6 |
| ÑAM                  |             4 |                  3 |
| ñññí                 |             8 |                  4 |
| Ñam3                 |             5 |                  4 |
| ÑAAAAM               |             7 |                  6 |
| ñññññ                |            10 |                  5 |
| eeeeeñí              |             9 |                  7 |
| bbbbbbbññññíóo       |            20 |                 14 |
+----------------------+---------------+--------------------+
Comment 2 Katrin Fischer 2015-05-15 22:23:25 UTC
Hi Pablo,
can you check if your framework includes the controlfield for setting the encoding for the MARC record? - I think it's LDR, pos. 9 Make sure that it's also linked to the standard plugin for the leader.
Comment 3 pablo.jimenez 2015-05-18 06:57:10 UTC
(In reply to Katrin Fischer from comment #2)
> Hi Pablo,
> can you check if your framework includes the controlfield for setting the
> encoding for the MARC record? - I think it's LDR, pos. 9 Make sure that it's
> also linked to the standard plugin for the leader.

Hi Katrin.

The field exists, in 000 tag, and its asociated with marc21_leader.pl.

An example of leader tax in marcxml in the database is:
<leader>00147    a2200085   4500</leader>

The 'a' in the pos. 09 is what you asked for?

Also, tags for title as 100a or 245a show
<datafield tag="100" ind1=" " ind2=" ">
    <subfield code="a">Přez</subfield>           --> It should be Pérez
  </datafield>
  <datafield tag="245" ind1=" " ind2=" ">
    <subfield code="a">Jimňez Muǫz</subfield>    --> It should be Jiménez Muñoz
  </datafield>

Thanks for your time.

Pablo
Comment 4 Katrin Fischer 2015-05-19 08:45:37 UTC
Hm, strange. Just to make sure - does the problem also appear in the default framework?
Comment 5 Owen Leonard 2016-06-21 15:28:47 UTC
No activity on this bug for over a year. I think we should assume it is resolved. Please reopen if you have additional information to help us reproduce the problem.