When using the basic editor for cataloging, entry of new records, fields that contain UTF-8 are silent ignored when saved. When that record is reviewed the field that had contained one of more UTF-8 character is now blank. When an item for that record is entered, fields that contain UTF-8 work fine. A clue may be that when a new record is created, or changed with some UTF-8 in the title field 245a, the field appears to be blank, and if the save button is hit again without making any changes, there is a complaint “Can’t save this record because the following field aren’t filled, tag 245 subfield a Title in tab 0” that you can’t save with a blank title field. However the display of the record that shows the tile and the record number shows the title correctly including the UTF-8 characters exactly as typed in the title. This problem occurs in Koha 18.11 and 21.05. Both are running on Ubuntu 18.11 and now on 20.04.3 LTS. I have tested that mysql tables will accept input from a mysql prompt in UTF-8, and displays it correctly in a query. Have tested that a short HTML file can be read by Apach2 and displays correctly. I have tested that Ubuntu’s locale is set to support UTF-8, by running the locale command. I am using Koha in Canada, using English. I tried the advanced editor option. There was no change when I edited the same record.
Created attachment 125342 [details] log from entering a field with UTF-8 The log contents after 1 try to enter a value in record with field 245,a containing several UTF-8 charcaters.
Hi Courtenay, please check your framework configuration and make sure you didn't remove/hide the LDR/000 field. This field will contain the encoding information on your record. If it's not present in a record, it will explain the problems you see with non-latin characters.
The suggestion that the leader field was the heart of the problem was correct. Thanks Katrin, this comment was very helpful. Therefore this was just a finger problem. However, I find the documentation on the topic of the frameworks and the use of plugins and how to write and debug them totally inadequate to non-existent.