Created attachment 130998 [details] screenshot of bug To recreate: - add Dutch "dut" to the AdvancedSearchLanguages option - go to the advanced search page /cgi-bin/koha/opac-search.pl - click open the Language select menu - note that the text for Dutch is "nedərlɑns (Dutch)" It should render as "nedərlɑns (Dutch)" but instead the numerical character references for the fourth and seventh characters are rendered literally, as if they were escaped. I haven't looked into this further so I don't know _why_ this is happening but Koha's OPAC uses charset=utf-8 so I think we should be able to simply use the actual characters without any references. Perhaps it's a problem with the encoding of strings in the database? But when I run a report to pull all the rows from the `language_descriptions` table, the word "nedərlɑns" renders correctly.
Hi Eric, I see "Nederlands (Dutch)". What was your query to the language_descriptions table? For me it looks like this: MariaDB [koha_kohadev]> select * from language_descriptions where subtag = "nl"; +--------+----------+------+-----------------+-----+ | subtag | type | lang | description | id | +--------+----------+------+-----------------+-----+ | nl | language | de | Niederländisch | 148 | | nl | language | en | Dutch | 146 | | nl | language | fr | Néerlandais | 147 | | nl | language | nl | Nederlands | 145 | | NL | region | en | Netherlands | 319 | | NL | region | nl | Nederland | 320 | +--------+----------+------+-----------------+-----+
I just used a report `select * FROM language_descriptions` and found the Dutch entries further down. If I run your query I get this: subtag type lang description id nl language de Niederländisch 138 nl language en Dutch 26 nl language fr Néerlandais 27 nl language nl nedərlɑns 25 NL region en Netherlands 299 NL region nl Nederland 300 Should we change the "nedərlɑns" description in the database to "Nederlands" then? It displays right in the reports results, the only problem is on the advanced search page. I see the same results you've shown on koha-testing-docker so perhaps this was changed at some point?
(In reply to Eric Phetteplace from comment #2) > I just used a report `select * FROM language_descriptions` and found the > Dutch entries further down. If I run your query I get this: > > subtag type lang description id > nl language de Niederländisch 138 > nl language en Dutch 26 > nl language fr Néerlandais 27 > nl language nl nedərlɑns 25 > NL region en Netherlands 299 > NL region nl Nederland 300 > > Should we change the "nedərlɑns" description in the database to "Nederlands" > then? It displays right in the reports results, the only problem is on the > advanced search page. I see the same results you've shown on > koha-testing-docker so perhaps this was changed at some point? I am not sure if we can trust the report here... what you see here are nedərlɑns HTML-entities that would look correct in the browser as it renders them (unless they are escaped, as it probably happens on the advanced search. Could you try and download the CSV export to compare? About the right naming I am not sure, but adding Marcel for maybe some help.
You're correct, it's stored in the database with HTML character references, but renders as you would want on the reports page. This is the CSV export's content from our live Koha instance (21.05.09.000): subtag,type,lang,description,id nl,language,de,Niederländisch,138 nl,language,en,Dutch,26 nl,language,fr,Néerlandais,27 nl,language,nl,nedərlɑns,25 NL,region,en,Netherlands,299 NL,region,nl,Nederland,300 So the question is why the character references aren't interpreted in the <select> menu on the advanced search page. All I could think to check is the encoding, but I see that both pages have "<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />" so I'm stumped.
I believe the reason is that we are using the | html filter in the select - which is as it should be and escapes any code that might be there. The problem is in the database, as you said earlier we should just store the name in utf-8 and without the entities. Do you have access to your database to fix this with SQL? We might still want to look into why it was not updated when it was for others - it seems ok on newer installations, but that could take a while.
(In reply to Katrin Fischer from comment #5) > I believe the reason is that we are using the | html filter in the select - > which is as it should be and escapes any code that might be there. The > problem is in the database, as you said earlier we should just store the > name in utf-8 and without the entities. Do you have access to your database > to fix this with SQL? > > We might still want to look into why it was not updated when it was for > others - it seems ok on newer installations, but that could take a while. Our instance is hosted by ByWater but I'll reach out to them and link to this bug report.
(In reply to Eric Phetteplace from comment #2) > I just used a report `select * FROM language_descriptions` and found the > Dutch entries further down. If I run your query I get this: > > subtag type lang description id > nl language de Niederländisch 138 > nl language en Dutch 26 > nl language fr Néerlandais 27 > nl language nl nedərlɑns 25 > NL region en Netherlands 299 > NL region nl Nederland 300 > > Should we change the "nedərlɑns" description in the database to "Nederlands" > then? Yes, Nederlands is the correct spelling. Change it. The first description seems to be a fonetic one. But apart from replacing it, I would say that there is some encoding issue too. I already have this entry: | nl | language | nl | Nederlands | 25 |
Should we create a just-in-case database update here in case there are more old installations with the wrong description?
(In reply to Katrin Fischer from comment #8) > Should we create a just-in-case database update here in case there are more > old installations with the wrong description? Good plan