Bug 31640

Summary: Fuzzy translations of preferences can cause missing sections and inaccurate translations
Product: Koha Reporter: Kevin Carnes <kevin.carnes>
Component: I18N/L10NAssignee: Kevin Carnes <kevin.carnes>
Status: RESOLVED FIXED QA Contact: Testopia <testopia>
Severity: normal    
Priority: P5 - low CC: bgkriegel, caroline.cyr-la-rose, dubyk, f.demians, jonathan.druart
Version: master   
Hardware: All   
OS: All   
Change sponsored?: --- Patch complexity: ---
Documentation contact: Documentation submission:
Text to go in the release notes:
Version(s) released in:
23.05.00,22.11.05
Attachments: This patch ignores fuzzy translations for preferences and warns if there are multiple sections with the same translated name.
Bug 31640: Fuzzy translations of preferences can cause missing sections and inaccurate translations
Bug 31640: Fuzzy translations of preferences can cause missing sections and inaccurate translations
Bug 31640: Fuzzy translations of preferences can cause missing sections and inaccurate translations
Bug 31640: Fuzzy translations of preferences can cause missing sections and inaccurate translations
Bug 31640: Fuzzy translations of preferences can cause missing sections and inaccurate translations
Bug 31640: Fuzzy translations of preferences can cause missing sections and inaccurate translations

Description Kevin Carnes 2022-09-28 08:37:18 UTC
We noticed that when installing the Swedish translations in 22.05, the translate program was using fuzzy translations for preferences. This resulted in many strange translations when viewing the preferences in Swedish.

There were also some sections that were missing since multiple sections had the same translated name and there can only be one section for each name since they are hash keys.

Our solution is to ignore fuzzy translations for preferences, since we prefer the original English, but it can be made an option if the fuzzy translations are helpful to others.

The solution also adds warnings if a translation causes multiple sections to have the same name.

When I run the warnings with fuzzy translations in 22.05.03, I get warnings in 65 languages (am-Ethi, ar-Arab, az-AZ, be-BY, bg-Cyrl, bn-IN, ca-ES, cs-CZ, cy-GB, da-DK, de-CH, el-GR, en-GB, en-NZ, eo, eu, fa-Arab, fr-CA, gl, he-Hebr, hi, hr-HR, hu-HU, hyw-Armn, id-ID, is-IS, ja-Jpan-JP, ka, ko-Kore-KP, ku-Arab, lo-Laoo, mi-NZ, mon, mr, ms-MY, nb-NO, ne-NE, nl-BE, nl-NL, nn-NO, oc, pbr, pl-PL, prs, pt-BR, pt-PT, ro-RO, ru-RU, sd-PK, sk-SK, sl-SI, sq-AL, sr-Cyrl, sv-SE, ta-LK, te, tet, th-TH, tl-PH, tr-TR, uk-UA, ur-Arab, vi-VN, zh-Hans-CN, zh-Hant-TW).

If there are languages where multiple sections should have the save name, we can discuss using an array instead of a hash for the sections.
Comment 1 Kevin Carnes 2022-09-28 09:08:49 UTC Comment hidden (obsolete)
Comment 2 Kevin Carnes 2022-09-28 09:11:26 UTC Comment hidden (obsolete)
Comment 3 Kevin Carnes 2022-09-28 09:12:44 UTC Comment hidden (obsolete)
Comment 4 Kevin Carnes 2022-09-28 09:17:58 UTC Comment hidden (obsolete)
Comment 5 Kevin Carnes 2022-09-28 09:50:04 UTC
I couldn't reproduce the error with "koha-translate -i sv-SE", but realized that I had an old docker image with Koha 21 in /usr/share. I updated the image and can reproduce the error, but Circulation is now translated as Utlåning in Swedish.

If you would like, I can attach a log of all of the conflicting sections for all languages so that you can test a language you are more comfortable with.

For example en-GB:
In file "cataloguing.pref", "cataloguing the record" is a translation for sections "Record structure", "Interface" at /kohadevbox/koha/misc/translator/LangInstaller.pm line 184.
In file "enhanced_content.pref", "Enhanced content settings" is a translation for sections "Amazon", "Library Thing", "Tagging", "Google", "Syndetics", "All", "OverDrive", "Manual" at /kohadevbox/koha/mi
sc/translator/LangInstaller.pm line 184.
In file "circulation.pref", " Self return module" is a translation for sections "Self check-in module", "Stock rotation module" at /kohadevbox/koha/misc/translator/LangInstaller.pm line 184.
In file "circulation.pref", "Returns Policy" is a translation for sections "Recalls", "Return claims", "Fines Policy" at /kohadevbox/koha/misc/translator/LangInstaller.pm line 184.
In file "circulation.pref", " Self Issue" is a translation for sections "Interface", "Interlibrary loans" at /kohadevbox/koha/misc/translator/LangInstaller.pm line 184.
In file "circulation.pref", " Batch issue" is a translation for sections "Article requests", "Batch checkout" at /kohadevbox/koha/misc/translator/LangInstaller.pm line 184.
In file "circulation.pref", " Self issue module" is a translation for sections "Self check-out module", "Housebound module" at /kohadevbox/koha/misc/translator/LangInstaller.pm line 184.
In file "web_services.pref", " Kohas" is a translation for sections "IdRef", "Reporting", "ILS-DI", "Mana KB", "General", "REST API", "OAI-PMH" at /kohadevbox/koha/misc/translator/LangInstaller.pm line 1
84.
In file "patrons.pref", "User forms" is a translation for sections "General", "Security", "Privacy", "Patron forms" at /kohadevbox/koha/misc/translator/LangInstaller.pm line 184.
In file "tools.pref", " User cards" is a translation for sections "News", "Batch item", "Patron cards", "Barcodes", "Upload" at /kohadevbox/koha/misc/translator/LangInstaller.pm line 184.
Comment 6 Kevin Carnes 2022-09-28 11:12:48 UTC
Created attachment 141050 [details] [review]
Bug 31640: Fuzzy translations of preferences can cause missing sections and inaccurate translations

This patch ignores fuzzy translations for preferences and warns if there are multiple sections with the same translated name.

Test Plan:
1)  Install English United Kingdom translations (./misc/translator/translate install en-GB)
2)  Go to Koha administration in the staff interface
3)  Click Global system preferences
4)  Select I18N/L10N preferences
5)  Enable English United Kingdom in the language preference for staff interface
6)  Save all I18N/L10N preferences
7)  Return to Koha administration
8)  Select English United Kingdom as the language at the bottom of the screen
9)  Click on Global system preferences
10) Select Circulation
11) Observe that there is only SelfCheckInMainUserBlock or StockRotation, but not both
12) Apply the patch
13) Install English United Kingdom translations (./misc/translator/translate install en-GB)
14) Go to Koha administration
15) Select English United Kingdom as the language at the bottom of the screen
16) Click on Global system preferences
17) Select Circulation
18) Observe that SelfCheckInMainUserBlock and StockRotation are both present
Comment 7 Kevin Carnes 2022-09-28 11:19:29 UTC
Of course it makes more sense to test it in English. The bug can be reproduced with koha-translate, but the patch won't affect the LangInstaller.pm in /usr/share on docker, so it's best to run the translate program in the repository.
Comment 8 Caroline Cyr La Rose 2023-01-10 15:09:06 UTC
Created attachment 145192 [details] [review]
Bug 31640: Fuzzy translations of preferences can cause missing sections and inaccurate translations

This patch ignores fuzzy translations for preferences and warns if there are multiple sections with the same translated name.

Test Plan:
1)  Install English United Kingdom translations (./misc/translator/translate install en-GB)
2)  Go to Koha administration in the staff interface
3)  Click Global system preferences
4)  Select I18N/L10N preferences
5)  Enable English United Kingdom in the language preference for staff interface
6)  Save all I18N/L10N preferences
7)  Return to Koha administration
8)  Select English United Kingdom as the language at the bottom of the screen
9)  Click on Global system preferences
10) Select Circulation
11) Observe that there is only SelfCheckInMainUserBlock or StockRotation, but not both
12) Apply the patch
13) Install English United Kingdom translations (./misc/translator/translate install en-GB)
14) Go to Koha administration
15) Select English United Kingdom as the language at the bottom of the screen
16) Click on Global system preferences
17) Select Circulation
18) Observe that SelfCheckInMainUserBlock and StockRotation are both present

Signed-off-by: Caroline Cyr La Rose <caroline.cyr-la-rose@inlibro.com>
Comment 9 Caroline Cyr La Rose 2023-01-10 15:11:38 UTC
Thanks Kevin! I had also noticed this in fr-CA (it's a bit puzzling when you *know* there is a syspref called whatever, and you can't find it... it's like Koha is trying to gaslight you), so it's great that you've corrected it!
Comment 10 Jonathan Druart 2023-02-23 08:49:17 UTC
Do we really need to keep the warnings?
Comment 11 Katrin Fischer 2023-02-23 09:37:18 UTC
(In reply to Jonathan Druart from comment #10)
> Do we really need to keep the warnings?

What do you mean by warnings?
Comment 12 Katrin Fischer 2023-02-23 09:38:40 UTC
oh I see, in the patch:
      warn qq(In file "$file", "$translation" is a translation for sections ") . join('", "', @{$duplicates{$translation}}) . '"';

I was wondering if it was some kind of warning text generated within the files.
Comment 13 Kevin Carnes 2023-02-23 12:05:22 UTC
(In reply to Jonathan Druart from comment #10)
> Do we really need to keep the warnings?

No, it was mainly to see which languages were affected, but I thought it might also be helpful in the future if a translator happened to use the same translation for two sections or if an erroneous fuzzy translation was approved.

We had no idea it was a problem until we tried to find a certain section and were confused when sections would disappear after we updated the translation, so a warning would have been helpful, but not necessary.
Comment 14 Jonathan Druart 2023-03-22 13:18:13 UTC
Created attachment 148533 [details] [review]
Bug 31640: Fuzzy translations of preferences can cause missing sections and inaccurate translations

This patch ignores fuzzy translations for preferences and warns if there are multiple sections with the same translated name.

Test Plan:
1)  Install English United Kingdom translations (./misc/translator/translate install en-GB)
2)  Go to Koha administration in the staff interface
3)  Click Global system preferences
4)  Select I18N/L10N preferences
5)  Enable English United Kingdom in the language preference for staff interface
6)  Save all I18N/L10N preferences
7)  Return to Koha administration
8)  Select English United Kingdom as the language at the bottom of the screen
9)  Click on Global system preferences
10) Select Circulation
11) Observe that there is only SelfCheckInMainUserBlock or StockRotation, but not both
12) Apply the patch
13) Install English United Kingdom translations (./misc/translator/translate install en-GB)
14) Go to Koha administration
15) Select English United Kingdom as the language at the bottom of the screen
16) Click on Global system preferences
17) Select Circulation
18) Observe that SelfCheckInMainUserBlock and StockRotation are both present

Signed-off-by: Caroline Cyr La Rose <caroline.cyr-la-rose@inlibro.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Comment 15 Tomás Cohen Arazi 2023-04-06 12:30:20 UTC
Pushed to master for 23.05.

Nice work everyone, thanks!
Comment 16 Jacob O'Mara 2023-04-16 20:37:06 UTC
Nice work, thanks everyone!

Pushed to 22.11.x for the next release.