Bug 18688

Summary: Warnings about UTF-8 charset when creating a new language
Product: Koha Reporter: Marc Véron <veron>
Component: I18N/L10NAssignee: Bernardo Gonzalez Kriegel <bgkriegel>
Status: CLOSED FIXED QA Contact: Testopia <testopia>
Severity: minor    
Priority: P5 - low CC: 1joynelson, bgkriegel, f.demians, hayleypelham, lucas
Version: Main   
Hardware: All   
OS: All   
See Also: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18629
Change sponsored?: --- Patch complexity: Trivial patch
Documentation contact: Documentation submission:
Text to go in the release notes:
Version(s) released in:
20.05.00, 19.11.02, 19.05.07, 18.11.13
Attachments: Bug 18688: Warnings about UTF-8 charset when creating a new language
Bug 18688: Warnings about UTF-8 charset when creating a new language
Bug 18688: Warnings about UTF-8 charset when creating a new language

Description Marc Véron 2017-05-28 13:10:30 UTC
Creatinng a new language produces several warnings.

To reproduce:
cd /koha/misc/translator$ 
perl translate create aa-AA 

Or, if env vars are not set (change path as appropriate):
sudo env KOHA_CONF=/etc/koha/sites/kohadev/koha-conf.xml
PERL5LIB=/home/vagrant/kohaclone perl translate create aa-AA


Result:

>Warning: Can't determine original templates' charset, defaulting to UTF-8
>Warning: Charset Out defaulting to UTF-8
>Removing empty file /home/vagrant/kohaclone/misc/translator/po/aa-AA-staff-prog.po
>I UTF-8 O UTF-8 at /home/vagrant/kohaclone/misc/translator/tmpl_process3.pl line 345.
Comment 1 Bernardo Gonzalez Kriegel 2019-12-10 00:17:16 UTC
Created attachment 96122 [details] [review]
Bug 18688: Warnings about UTF-8 charset when creating a new language

This patch removes some warnings when creating new languages,
those related with uft8 and two not related.

Tipical messages are:
>Warning: Can't determine original templates' charset, defaulting to UTF-8
>Warning: Charset Out defaulting to UTF-8
>...
>Removing empty file /../xx-YY-marc-NORMARC.po
>I UTF-8 O UTF-8 at /../kohaclone/misc/translator/tmpl_process3.pl line 365.
>Use of uninitialized value $pref_name in concatenation (.) or string at /.../misc/translator/LangInstaller.pm line 197.
>Created /../misc/translator/po/xx-YY-messages.po.

First four are 'normal' messages
The fifth one correspond to an unquoted preference string
The sixth one is a normal output message from msgint command
We will hide them unless verbose '-v' option is selected,
and fix the error.

To test:
1) Go to misc/translator
2) Create new language files
  $ perl translate create xx-YY
3) Check warnings and messages
4) Apply the patch
5) Remove created files, and create them again
   Ther must be no messages
6) Repeat 5) with verbose option '-v', check
   old messages

7) BONUS: without the patch update a language,
   for example de-DE:
   $ perl translate update de-DE
   Note (git status) a new file, po/de-DE-messages.po~
   This is a backup file from msgmerge command
   With the patch this backup is removed.
Comment 2 Jonathan Druart 2019-12-16 09:02:30 UTC
Created attachment 96305 [details] [review]
Bug 18688: Warnings about UTF-8 charset when creating a new language

This patch removes some warnings when creating new languages,
those related with uft8 and two not related.

Tipical messages are:
>Warning: Can't determine original templates' charset, defaulting to UTF-8
>Warning: Charset Out defaulting to UTF-8
>...
>Removing empty file /../xx-YY-marc-NORMARC.po
>I UTF-8 O UTF-8 at /../kohaclone/misc/translator/tmpl_process3.pl line 365.
>Use of uninitialized value $pref_name in concatenation (.) or string at /.../misc/translator/LangInstaller.pm line 197.
>Created /../misc/translator/po/xx-YY-messages.po.

First four are 'normal' messages
The fifth one correspond to an unquoted preference string
The sixth one is a normal output message from msgint command
We will hide them unless verbose '-v' option is selected,
and fix the error.

To test:
1) Go to misc/translator
2) Create new language files
  $ perl translate create xx-YY
3) Check warnings and messages
4) Apply the patch
5) Remove created files, and create them again
   Ther must be no messages
6) Repeat 5) with verbose option '-v', check
   old messages

7) BONUS: without the patch update a language,
   for example de-DE:
   $ perl translate update de-DE
   Note (git status) a new file, po/de-DE-messages.po~
   This is a backup file from msgmerge command
   With the patch this backup is removed.

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Comment 3 Katrin Fischer 2019-12-22 10:00:24 UTC
Created attachment 96565 [details] [review]
Bug 18688: Warnings about UTF-8 charset when creating a new language

This patch removes some warnings when creating new languages,
those related with uft8 and two not related.

Tipical messages are:
>Warning: Can't determine original templates' charset, defaulting to UTF-8
>Warning: Charset Out defaulting to UTF-8
>...
>Removing empty file /../xx-YY-marc-NORMARC.po
>I UTF-8 O UTF-8 at /../kohaclone/misc/translator/tmpl_process3.pl line 365.
>Use of uninitialized value $pref_name in concatenation (.) or string at /.../misc/translator/LangInstaller.pm line 197.
>Created /../misc/translator/po/xx-YY-messages.po.

First four are 'normal' messages
The fifth one correspond to an unquoted preference string
The sixth one is a normal output message from msgint command
We will hide them unless verbose '-v' option is selected,
and fix the error.

To test:
1) Go to misc/translator
2) Create new language files
  $ perl translate create xx-YY
3) Check warnings and messages
4) Apply the patch
5) Remove created files, and create them again
   Ther must be no messages
6) Repeat 5) with verbose option '-v', check
   old messages

7) BONUS: without the patch update a language,
   for example de-DE:
   $ perl translate update de-DE
   Note (git status) a new file, po/de-DE-messages.po~
   This is a backup file from msgmerge command
   With the patch this backup is removed.

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Comment 4 Martin Renvoize 2019-12-23 12:09:25 UTC
Nice work everyone!

Pushed to master for 20.05
Comment 5 Joy Nelson 2020-01-12 19:02:59 UTC
Pushed to 19.11.x branch for 19.11.02
Comment 6 Lucas Gass 2020-01-13 22:50:59 UTC
backported to 19.05.x for 19.05.07
Comment 7 Hayley Pelham 2020-01-14 23:09:18 UTC
Backported to 18.11.x for 18.11.13