This source for example: http://www.webclarity.info/registry/display?fileID=69 All displays fine from basic editor - rancor however causes garbled characters
Created attachment 112526 [details] Garbled diacritics in Z39.50 results list
Created attachment 112527 [details] Garbled diacritics in MARC record
When performing a Z39.50 search from within the Advanced Cataloging Editor we often get results that display garbled diacritics. Sometimes changing the encoding from utf8 to MARC8 or vice versa can help, but often the diacritics don't display correctly with either encoding. We have to spend extra time editing these records to make sure we correct everything. It would improve accuracy and efficiency to have the diacritics display correctly. This isn't a problem when using Z39.50 to add a new record - it would be great if it worked the same way from within the Advanced Editor.
Hi Barbara, do you have an example target where the error occurs maybe?
Yes, it happens with Milwaukee, CARL, Marmot, Ohioink, Sacramento. In fact, it happens with most of them. There are only a few that display the diacritics correctly, such as Seattle or King County.
Hm, failing to find the Z39.50 data for these targets - and the ones I can find I am not sure are current and/or are missing information on the encoding.
I’ve listed several below. I realized that almost all of the ones that give bad diacritics are III catalogs, but I did find a non-III (Vancouver) that does the same thing. Seattle is an example of the correct diacritics. Milwaukee Host name: countycat.mcfls.org Port: 210 Database: INNOPAC Syntax: MARC21/USMARC Encoding: utf8 Ohiolink Host name: olc1.ohiolink.edu Port: 210 Database: INNOPAC Syntax: MARC21/USMARC Encoding: utf8 Sacramento Host name: www.saclibrarycatalog.org Port: 210 Database: INNOPAC Syntax: MARC21/USMARC Encoding: utf8 Vancouver Host name: z3950.vpl.ca Port: 210 Database: HORIZON Syntax: MARC21/USMARC Encoding: utf8 Seattle Host name: zserver.spl.org Port: 210 Database: HORIZON Syntax: MARC21/USMARC Encoding: MARC-8
Created attachment 112663 [details] [review] Bug 18051: Set UTF8 Flag for record searches/imports from rancor The advanced cataloging editor uses Koha::MetaSearcher to find and import records from Z3950. The records are correctly converted to UTF8, however, we need to set the flag in the record to ensure they are parsed correctly later To test: 1 - Add OHIOLINK as a z39 source as described in earlier comments 2 - Browse to Cataloging->Advanced editor 3 - Click Search->Advanced 4 - Select OHIOLINK 5 - Search for author 'capek karel' 6 - Note records look bad 7 - Apply patch 8 - Restart all the things 9 - Repeat 10 - Success!
Created attachment 112664 [details] [review] Bug 18051: Set UTF8 Flag for record searches/imports from rancor The advanced cataloging editor uses Koha::MetaSearcher to find and import records from Z3950. The records are correctly converted to UTF8, however, we need to set the flag in the record to ensure they are parsed correctly later To test: 1 - Add OHIOLINK as a z39 source as described in earlier comments 2 - Browse to Cataloging->Advanced editor 3 - Click Search->Advanced 4 - Select OHIOLINK 5 - Search for author 'capek karel' 6 - Note records look bad 7 - Apply patch 8 - Restart all the things 9 - Repeat 10 - Success! Signed-off-by: Barbara Johnson <barbara.johnson@bedfordtx.gov>
Success!! I tested records in multiple languages on Ohiolink as well as other Z39.50 servers and diacritics displayed correctly. I also tested Seattle which uses MARC-8 and had already been displaying the records correctly and they still do.
Thx Nick for taking this on and to Barbara for providing the extra information!
Created attachment 112804 [details] [review] Bug 18051: Set UTF8 Flag for record searches/imports from rancor The advanced cataloging editor uses Koha::MetaSearcher to find and import records from Z3950. The records are correctly converted to UTF8, however, we need to set the flag in the record to ensure they are parsed correctly later To test: 1 - Add OHIOLINK as a z39 source as described in earlier comments 2 - Browse to Cataloging->Advanced editor 3 - Click Search->Advanced 4 - Select OHIOLINK 5 - Search for author 'capek karel' 6 - Note records look bad 7 - Apply patch 8 - Restart all the things 9 - Repeat 10 - Success! Signed-off-by: Barbara Johnson <barbara.johnson@bedfordtx.gov> Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de> Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Pushed to master for 20.11, thanks to everybody involved!
backported to 20.05.x for 20.05.06
backported to 19.11.x for 19.11.12
Backported to 19.05.x branch for 19.05.17