Bug 29118 - Basic editor for new entry or editing records drops fields that contains any UTF-8 character
Summary: Basic editor for new entry or editing records drops fields that contains any ...
Status: CLOSED WONTFIX
Alias: None
Product: Koha
Classification: Unclassified
Component: Cataloging (show other bugs)
Version: 20.05
Hardware: PC Linux
: P5 - low normal
Assignee: Bugs List
QA Contact: Testopia
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2021-09-26 17:23 UTC by Courtenay Johnson
Modified: 2022-06-06 20:24 UTC (History)
1 user (show)

See Also:
GIT URL:
Initiative type: ---
Sponsorship status: ---
Comma delimited list of Sponsors:
Crowdfunding goal: 0
Crowdfunding committed: 0
Crowdfunding contact:
Patch complexity: ---
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:
Circulation function:


Attachments
log from entering a field with UTF-8 (2.07 KB, text/plain)
2021-09-26 17:46 UTC, Courtenay Johnson
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Courtenay Johnson 2021-09-26 17:23:57 UTC
When using the basic editor for cataloging, entry of new records, fields that contain UTF-8 are silent ignored when saved. When that record is reviewed the field that had contained one of more UTF-8 character is now blank.
When an item for that record is entered, fields that contain UTF-8 work fine.
A clue may be that when a new record is created, or changed with some UTF-8 in the title field 245a, the field appears to be blank, and if the save button is hit again without making any changes, there is a complaint “Can’t save this record because the following field aren’t filled, tag 245 subfield a Title in tab 0” that you can’t save with a blank title field. However the display of the record that shows the tile and the record number shows the title correctly including the UTF-8 characters exactly as typed in the title.
This problem occurs in Koha 18.11 and 21.05. Both are running on Ubuntu 18.11 and now on 20.04.3 LTS.

I have tested that mysql tables will accept input from a mysql prompt in UTF-8, and displays it correctly in a query.
 Have tested that a short HTML file can be read by Apach2 and displays correctly.
I have tested that Ubuntu’s locale is set to support UTF-8, by running the locale command.
I am using Koha in Canada, using English. 
I tried the advanced editor option. There was no change when I edited the same record.
Comment 1 Courtenay Johnson 2021-09-26 17:46:59 UTC
Created attachment 125342 [details]
log from entering a field with UTF-8

The log contents after 1 try to enter a value in record with field 245,a containing several UTF-8 charcaters.
Comment 2 Katrin Fischer 2021-09-26 21:44:44 UTC
Hi Courtenay, 

please check your framework configuration and make sure you didn't remove/hide the LDR/000 field. This field will contain the encoding information on your record. If it's not present in a record, it will explain the problems you see with non-latin characters.
Comment 3 Courtenay Johnson 2021-10-25 16:23:58 UTC
The suggestion that the leader field was the heart of the problem was correct. Thanks Katrin, this comment was very helpful.

Therefore this was just a finger problem. However, I find the documentation on the topic of the frameworks and the use of plugins and how to write and debug them totally inadequate to non-existent.