Bug 15395 - Internationalization: plural forms, context, and more
Summary: Internationalization: plural forms, context, and more
Status: Signed Off
Alias: None
Product: Koha
Classification: Unclassified
Component: I18N/L10N (show other bugs)
Version: master
Hardware: All All
: P5 - low enhancement (vote)
Assignee: Julian Maurice
QA Contact: Testopia
URL:
Whiteboard:
Keywords:
Depends on:
Blocks: 15522
  Show dependency treegraph
 
Reported: 2015-12-18 00:31 UTC by Julian Maurice
Modified: 2018-01-19 11:27 UTC (History)
9 users (show)

See Also:
Change sponsored?: ---
Patch complexity: Medium patch
Bot Control: ---
When did the bot last check this:
Who signed the patch off:
Text to go in the release notes:


Attachments
Bug 15395: Allow correct handling of plural translation (10.48 KB, patch)
2015-12-18 00:33 UTC, Julian Maurice
Details | Splinter Review
Bug 15395: Plural translations in templates (10.81 KB, patch)
2015-12-18 00:33 UTC, Julian Maurice
Details | Splinter Review
Bug 15395: Example usage of I18N Template::Toolkit plugin (2.42 KB, patch)
2015-12-18 00:33 UTC, Julian Maurice
Details | Splinter Review
Bug 15395: Add plural translation capabilities to JS files (107.28 KB, patch)
2016-01-06 16:46 UTC, Julian Maurice
Details | Splinter Review
Bug 15395: Allow correct handling of plural translation (10.48 KB, patch)
2016-04-07 13:24 UTC, Julian Maurice
Details | Splinter Review
Bug 15395: Plural translations in templates (10.81 KB, patch)
2016-04-07 13:24 UTC, Julian Maurice
Details | Splinter Review
Bug 15395: Example usage of I18N Template::Toolkit plugin (2.66 KB, patch)
2016-04-07 13:24 UTC, Julian Maurice
Details | Splinter Review
Bug 15395: Add plural translation capabilities to JS files (107.28 KB, patch)
2016-04-07 13:24 UTC, Julian Maurice
Details | Splinter Review
Bug 15395: Allow correct handling of plural translation (10.50 KB, patch)
2017-04-05 10:23 UTC, Julian Maurice
Details | Splinter Review
Bug 15395: Plural translations in templates (10.84 KB, patch)
2017-04-05 10:23 UTC, Julian Maurice
Details | Splinter Review
Bug 15395: Example usage of I18N Template::Toolkit plugin (2.69 KB, patch)
2017-04-05 10:23 UTC, Julian Maurice
Details | Splinter Review
Bug 15395: Add plural translation capabilities to JS files (107.30 KB, patch)
2017-04-05 10:23 UTC, Julian Maurice
Details | Splinter Review
Bug 15395: Exclude xslt from the list of themes when building locale_data.js (993 bytes, patch)
2017-04-06 08:13 UTC, Julian Maurice
Details | Splinter Review
Bug 15395: Fix PO header reading in po2json (1.65 KB, patch)
2017-04-06 08:13 UTC, Julian Maurice
Details | Splinter Review
Bug 15395: Fix output of msgcat (1.64 KB, patch)
2017-04-07 16:56 UTC, Julian Maurice
Details | Splinter Review
Bug 15395: Make Koha::I18N work with Plack + add tests (11.96 KB, patch)
2017-04-10 17:10 UTC, Julian Maurice
Details | Splinter Review
Bug 15395: Search .js files in more directories (1.36 KB, patch)
2017-04-11 12:56 UTC, Julian Maurice
Details | Splinter Review
Bug 15395: Allow correct handling of plural translation (10.61 KB, patch)
2017-04-18 19:41 UTC, Marc Véron
Details | Splinter Review
Bug 15395: Plural translations in templates (10.94 KB, patch)
2017-04-18 19:46 UTC, Marc Véron
Details | Splinter Review
Bug 15395: Add plural translation capabilities to JS files (107.39 KB, patch)
2017-04-18 19:47 UTC, Marc Véron
Details | Splinter Review
Bug 15395: Exclude xslt from the list of themes when building locale_data.js (1.09 KB, patch)
2017-04-18 19:48 UTC, Marc Véron
Details | Splinter Review
Bug 15395: Fix PO header reading in po2json (1.77 KB, patch)
2017-04-18 19:49 UTC, Marc Véron
Details | Splinter Review
Bug 15395: Fix output of msgcat (1.76 KB, patch)
2017-04-18 19:50 UTC, Marc Véron
Details | Splinter Review
Bug 15395: Make Koha::I18N work with Plack + add tests (12.08 KB, patch)
2017-04-18 19:51 UTC, Marc Véron
Details | Splinter Review
Bug 15395: Search .js files in more directories (1.49 KB, patch)
2017-04-18 19:53 UTC, Marc Véron
Details | Splinter Review
Bug 15395: Example usage of I18N Template::Toolkit plugin (2.92 KB, patch)
2017-04-18 19:57 UTC, Marc Véron
Details | Splinter Review
Bug 15395: Fix several issues in translation process (7.60 KB, patch)
2017-04-26 09:07 UTC, Julian Maurice
Details | Splinter Review
Bug 15395: Remove the use of Locale::Util's set_locale (1.62 KB, patch)
2017-08-11 10:44 UTC, Julian Maurice
Details | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Julian Maurice 2015-12-18 00:31:36 UTC
In response to Frédéric's comment on bug 15206 (http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=15206#c33) I wanted to see what could be done to handle translation of plural in Koha.
This is my attempt to manage translation in both Perl code and templates
Comment 1 Julian Maurice 2015-12-18 00:33:15 UTC Comment hidden (obsolete)
Comment 2 Julian Maurice 2015-12-18 00:33:18 UTC Comment hidden (obsolete)
Comment 3 Julian Maurice 2015-12-18 00:33:21 UTC Comment hidden (obsolete)
Comment 4 Julian Maurice 2016-01-06 16:46:11 UTC Comment hidden (obsolete)
Comment 5 Bernardo Gonzalez Kriegel 2016-04-07 13:02:38 UTC
Applying: Bug 15395: Example usage of I18N Template::Toolkit plugin
Using index info to reconstruct a base tree...
M       C4/Search.pm
M       koha-tmpl/intranet-tmpl/prog/en/modules/catalogue/results.tt
Falling back to patching base and 3-way merge...
Auto-merging koha-tmpl/intranet-tmpl/prog/en/modules/catalogue/results.tt
CONFLICT (content): Merge conflict in koha-tmpl/intranet-tmpl/prog/en/modules/catalogue/results.tt
Comment 6 Julian Maurice 2016-04-07 13:24:04 UTC Comment hidden (obsolete)
Comment 7 Julian Maurice 2016-04-07 13:24:08 UTC Comment hidden (obsolete)
Comment 8 Julian Maurice 2016-04-07 13:24:11 UTC Comment hidden (obsolete)
Comment 9 Julian Maurice 2016-04-07 13:24:15 UTC Comment hidden (obsolete)
Comment 10 Julian Maurice 2016-04-07 13:25:07 UTC
Patches rebased on master
Comment 11 Frédéric Demians 2016-06-03 08:11:28 UTC
./translate update fr-FR raises this error:

system call failed: /usr/bin/xgettext -L JavaScript
Comment 12 Julian Maurice 2016-06-15 06:27:02 UTC
(In reply to Frédéric Demians from comment #11)
> ./translate update fr-FR raises this error:
> 
> system call failed: /usr/bin/xgettext -L JavaScript

This error was due to a too old version of gettext, 0.19 is needed
Comment 13 Frédéric Demians 2016-06-15 06:50:18 UTC
(In reply to Julian Maurice from comment #12)

> This error was due to a too old version of gettext, 0.19 is needed

Yes. I can confirm that the whole set of patches works with the proper gettext version.
Comment 14 Julian Maurice 2016-06-15 07:06:15 UTC
(In reply to Frédéric Demians from comment #13)
> Yes. I can confirm that the whole set of patches works with the proper
> gettext version.
Hi Frédéric,

Did you follow all 3 test plans ?
If so, could you signoff these patches ?
Comment 15 Frédéric Demians 2016-06-15 09:17:15 UTC
I'm not over-confident with gettext.js. It seems quiet old and unmaintained. Have you ever considered this:

https://slexaxton.github.io/Jed/

I'd be also interested in feedback from Bernardo, since this patch will modify the translation workflow.
Comment 16 Marc Véron 2017-04-05 09:10:15 UTC
Patch does no longer apply.

How would disambiguation work?

e.g. for the 3 meanings of "Title" mentioned in
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11285#c25
Comment 17 Julian Maurice 2017-04-05 10:23:00 UTC Comment hidden (obsolete)
Comment 18 Julian Maurice 2017-04-05 10:23:04 UTC Comment hidden (obsolete)
Comment 19 Julian Maurice 2017-04-05 10:23:08 UTC Comment hidden (obsolete)
Comment 20 Julian Maurice 2017-04-05 10:23:13 UTC Comment hidden (obsolete)
Comment 21 Julian Maurice 2017-04-05 10:29:37 UTC
(In reply to Marc Véron from comment #16)
> Patch does no longer apply.
> 
> How would disambiguation work?
> 
> e.g. for the 3 meanings of "Title" mentioned in
> https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11285#c25

To attach context to a string, you have to use one of the 'p' variants (__p, __px, __np, __npx). This context will appear in the PO file to help the translator and gettext will use this information to choose the right translation

https://www.gnu.org/software/gettext/manual/html_node/Contexts.html

Eg.
  __p('Patron', 'Title')
  __p('Bibliographic record', 'Title')
Comment 22 Marc Véron 2017-04-05 20:14:44 UTC
I tried to test  "Example usage of I18N Template::Toolkit plugin" , using the test plan from comment #7

After perl translate update fr-FR, I edited: 
koha/misc/translator/po/fr_FR/LC_MESSAGES/Koha.po
translated the strings and changed the line:
"Plural-Forms: nplurals=INTEGER; plural=EXPRESSION;\n"
to:
"Plural-Forms: nplurals=2; plural=(n > 1);\n"

However, perl translate install fr-FR complains with following warnings:

Use of uninitialized value $pref_name in concatenation (.) or string at LangInstaller.pm line 202.
Use of uninitialized value $pref_name in concatenation (.) or string at LangInstaller.pm line 244.
NO PLURAL FORM HEADER FOUND - DEFAULTING TO 2
print() on closed filehandle $fh at LangInstaller.pm line 689.

What did I miss?
Comment 23 Julian Maurice 2017-04-06 08:13:46 UTC Comment hidden (obsolete)
Comment 24 Julian Maurice 2017-04-06 08:13:50 UTC Comment hidden (obsolete)
Comment 25 Julian Maurice 2017-04-06 08:15:22 UTC
(In reply to Marc Véron from comment #22)
> NO PLURAL FORM HEADER FOUND - DEFAULTING TO 2

This was caused by an incompatibility between po2json and the newer version of Locale::PO.
This is fixed in the last patch.
Comment 26 Marc Véron 2017-04-06 09:29:50 UTC
Hi Julian, I can not yet make it work. I think I missed something

perl translate install fr-FR
Use of uninitialized value $pref_name in concatenation (.) or string at LangInstaller.pm line 202.
Use of uninitialized value $pref_name in concatenation (.) or string at LangInstaller.pm line 244.
SKIPPING ERROR MARKER IN HEADER: #-#-#-#-#  Koha-perl.pot (PACKAGE VERSION)  #-#-#-#-#
SKIPPING ERROR MARKER IN HEADER: #-#-#-#-#  Koha-js.pot (PACKAGE VERSION)  #-#-#-#-#

Vales are not replaced in .tt file:
 [% SEARCH_RESULT.items_count %] [% tn('item', 'items', SEARCH_RESULT.items_count) %](...)
I would expect:
 [% SEARCH_RESULT.items_count %] [% tn('exemplaire', 'exemplaires', SEARCH_RESULT.items_count) %](...)


This is the .po file after editing Plural forms:

------------
# FIRST AUTHOR <EMAIL@ADDRESS>, YEAR.
#
#, fuzzy
msgid ""
msgstr ""
"Project-Id-Version: PACKAGE VERSION\n"
"Report-Msgid-Bugs-To: \n"
"POT-Creation-Date: 2017-04-06 10:52+0200\n"
"PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
"Last-Translator: FULL NAME <EMAIL@ADDRESS>\n"
"Language-Team: LANGUAGE <LL@li.org>\n"
"Language: \n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=UTF-8\n"
"Content-Transfer-Encoding: 8bit\n"
"#-#-#-#-#  Koha-perl.pot (PACKAGE VERSION)  #-#-#-#-#\n"
"Plural-Forms: nplurals=2; plural=(n > 1);\n"
"#-#-#-#-#  Koha-js.pot (PACKAGE VERSION)  #-#-#-#-#\n"

#: koha-tmpl/intranet-tmpl/prog/en/modules/catalogue/results.tt:1
msgid "item"
msgid_plural "items"
msgstr[0] "exemplaire"
msgstr[1] "exemplaires"
---------------
Comment 27 Julian Maurice 2017-04-06 09:42:02 UTC
(In reply to Marc Véron from comment #26)
> Hi Julian, I can not yet make it work. I think I missed something
> 
> perl translate install fr-FR
> Use of uninitialized value $pref_name in concatenation (.) or string at
> LangInstaller.pm line 202.
> Use of uninitialized value $pref_name in concatenation (.) or string at
> LangInstaller.pm line 244.
> SKIPPING ERROR MARKER IN HEADER: #-#-#-#-#  Koha-perl.pot (PACKAGE VERSION) 
> #-#-#-#-#
> SKIPPING ERROR MARKER IN HEADER: #-#-#-#-#  Koha-js.pot (PACKAGE VERSION) 
> #-#-#-#-#

I also have these messages, but they are only warnings, we can fix them later

> 
> Vales are not replaced in .tt file:
>  [% SEARCH_RESULT.items_count %] [% tn('item', 'items',
> SEARCH_RESULT.items_count) %](...)
> I would expect:
>  [% SEARCH_RESULT.items_count %] [% tn('exemplaire', 'exemplaires',
> SEARCH_RESULT.items_count) %](...)
> 

This result is expected. Messages are not translated directly in template files, but tn() will return the correct translation at runtime.
Comment 28 Julian Maurice 2017-04-07 16:56:16 UTC Comment hidden (obsolete)
Comment 29 Julian Maurice 2017-04-10 17:10:30 UTC Comment hidden (obsolete)
Comment 30 Marc Véron 2017-04-11 08:11:24 UTC
Hi Julian

I tried again with the tt example from patch  'Bug 15395: Example usage of I18N Template::Toolkit plugin'

Followed the test plan (with fr-FR and de-DE), the translations for item/items showed up as expected (exemplaire/exemplaires and Exemplar/Exemplare). Great!

However it seems that the second part of line 630 in results.tt is no longer picked for translation or does not translate properly (n available rsp. None available). I have the impression that the rest of the line after the plural stuff is skipped.
Comment 31 Julian Maurice 2017-04-11 08:31:35 UTC
(In reply to Marc Véron from comment #30)
> However it seems that the second part of line 630 in results.tt is no longer
> picked for translation or does not translate properly (n available rsp. None
> available). I have the impression that the rest of the line after the plural
> stuff is skipped.

This is expected, quite annoying, but expected. Due to the way the template strings extractor works (not the new one brought by this bug but the current one), almost every change in a template will result in a change in the PO file.

For instance,
  msgid "%s %s %sitems%sitem%s%s, %s available:%s, None available%s %s "
will be changed to
  msgid "%s %s %s%s, %s available:%s, None available%s %s "
and will be marked as "fuzzy" so it will not be used.

But you can fix the translation, remove the fuzzy flag, and run ./translate install. Text will be translated again.
Comment 32 Marc Véron 2017-04-11 09:06:23 UTC
Ah, got it, thanks for the explanation :-)
Comment 33 Marc Véron 2017-04-11 10:22:22 UTC
Hmm, I can not make js files translate.

What I did:

Change staff-global.js as suggested in comment #9

function openHelp(){
     alert(__nx("There is one thing", "There are {count} things", 3, {count: 3}));

The alert shows up fine in English version (Hit the 'Help' link)

However after translate update, I find nothing in Koha.po (only the strings from the tt example)
Comment 34 Julian Maurice 2017-04-11 11:51:10 UTC
Ah, you found a bug! :-)
Actually the string extractor only search in koha-tmpl/**/en/ to avoid picking strings from already translated files (when I wrote the test plan, staff-global.js was inside /en/ I think).

I guess I need to refine the file search to also search in koha-tmpl/intranet-tmpl/prog/js and koha-tmpl/opac-tmpl/bootstrap/js (anything else ?).

Anyway, to test it right now you could modify a .js file in the /en/ directory
Comment 35 Julian Maurice 2017-04-11 12:56:29 UTC Comment hidden (obsolete)
Comment 36 Marc Véron 2017-04-11 15:25:15 UTC
OK, I tested with 3 files:

koha-tmpl/intranet-tmpl/prog/en/js/categories.js
koha-tmpl/intranet-tmpl/prog/js/staff-global.js
koha-tmpl/opac-tmpl/bootstrap/js/global.js

All files were picked up in Koha.po

I translated them and did a translate install

Example for JavaScript used (added as line 1 of koha-tmpl/intranet-tmpl/prog/en/js/categories.js):

alert(__nx("There is one category", "There are {count} categories", 4, {count: 4}));

Translation in misc/translator/po/de_DE/LC_MESSAGES/Koha.po:

#: koha-tmpl/intranet-tmpl/prog/en/js/categories.js:1
msgid "There is one category"
msgid_plural "There are {count} categories"
msgstr[0] "Es gibt {count} Kategorien"
msgstr[1] "Es gibt nur eine Kategorie..."

English page displays "There are 4 categories"  (OK)
German page displays "Es gibt nur eine Kategorie..." (WRONG)
Comment 37 Julian Maurice 2017-04-11 15:47:51 UTC
(In reply to Marc Véron from comment #36)
> German page displays "Es gibt nur eine Kategorie..." (WRONG)

It depends on how you configured the Plural-Forms header, but by convention, msgstr[0] is the singular form and msgstr[1] is the plural form, so it looks ok to me
Comment 38 Marc Véron 2017-04-18 19:33:54 UTC
Oh, reading helps...
Comment 39 Marc Véron 2017-04-18 19:41:05 UTC
Created attachment 62326 [details] [review]
Bug 15395: Allow correct handling of plural translation

Locale::Maketext does not allow correct handling of plural translation
for languages that have more than one plural forms.
Locale::Messages does.

So Koha::I18N is now a wrapper around Locale::Messages, just like
Locale::TextDomain, and export the same symbols as Locale::TextDomain.
You can refer to documentation of Locale::TextDomain to know how to use
exported subroutines.

The PO file moves from misc/translator/po/xx-XX-messages.po to
misc/translator/po/xx_XX/LC_MESSAGES/Koha.po and now needs to be
compiled to MO in order to be used by Koha.
Compilation of PO file is done by running:
  ./translate install xx-XX

Remove dependency to Locale::Maketext and Locale::Maketext::Lexicon
Add dependency to Locale::Messages

Test plan:
1. Open a .pl script or .pm module with your favorite text editor
2. Add 'use Koha::I18N;' in the beginning of file
3. Use one of the subroutines exported by Koha::I18N and be sure to have
   a way to visualize the result (pass result to the template for
   example, or simply warn and watch the log file)
4. cd misc/translator && ./translate update fr-FR # try other languages
5. Open misc/translator/po/fr_FR/LC_MESSAGES/Koha.po and translate your
   string(s)
   You will need to change the "Plural-Forms" header. See
   https://localization-guide.readthedocs.org/en/latest/l10n/pluralforms.html
6. ./translate install fr-FR
7. Use your web browser to go to the page that should display the
   translation, change language and verify the translation is correct

Example usage:
  __("Hi")
  __x("Hi {name}", name => 'Bob')
  __n("item", "items", $num_items)
  __nx("one item", "{count} items", $num_items, count => $num_items)
  __p("Bibliographic record", "item")

Signed-off-by: Marc Véron <veron@veron.ch>
Comment 40 Marc Véron 2017-04-18 19:46:12 UTC
Created attachment 62327 [details] [review]
Bug 15395: Plural translations in templates

Provides a way to handle translation of plural forms in templates

Add Template::Toolkit plugin for Koha::I18N

Use like this:

[%# USE the plugin and define some macros %]
[% PROCESS 'i18n.inc' %]

[%# tn is the equivalent of __n %]
[%# macro names can't start with underscore, t is for "translate" %]
[% tn('item', 'items', num_items) %]

Extraction of strings from templates is a bit complicated and use
Template::Parser and PPI. Template is compiled into Perl code and then
analyzed by PPI. It is slow, but should be correct even with complex
constructions.

Add dependency to PPI

Test plan:
1. Open a template file (.tt or .inc) with your favorite text editor
2. Add the PROCESS directive mentioned above in the beginning of file
3. Use one of the t* macros defined in i18n.inc. They are used like
   their "__" equivalent, with one difference: the 'x' variants take a
   hashref instead of a hash as last parameter
4. cd misc/translator && ./translate update fr-FR
5. Open misc/translator/po/fr_FR/LC_MESSAGES/Koha.po and translate your
   string(s)
6. ./translate install fr-FR
7. Use your web browser to go to the page that should display the
   translation, change language and verify the translation is
   correct

Signed-off-by: Marc Véron <veron@veron.ch>
Comment 41 Marc Véron 2017-04-18 19:47:28 UTC
Created attachment 62328 [details] [review]
Bug 15395: Add plural translation capabilities to JS files

It adds Javascript equivalent of Koha::I18N's exported subroutines, and
they are used the same way.

String extraction is done only on *.js files and require gettext 0.19
(available in Debian stable, and also in wheezy-backports)

It adds Javascript library Gettext.js for handling translation and a
Perl script po2json to transform PO file into JSON.

Gettext.js and po2json both come from Locale::Simple.
There are several tools named po2json. It's simpler to integrate this
one into Koha than to check if the good one is installed on the system.
Locale::Simple is not needed.

To avoid polluting the global namespace too much, this patch also
introduce a global JS object named Koha and add some stuff in Koha.i18n

Test plan:
1. Add a translatable string in a JS file. For example, add this:
     alert(__nx("There is one item", "There are {count} items", 3,
     {count: 3}));
   to staff-global.js
2. cd misc/translator && ./translate update fr-FR
3. Open misc/translator/po/fr_FR/LC_MESSAGES/Koha.po, verify that your
   string is present, and translate it
4. cd misc/translator && ./translate install fr-FR
5. (Optional) Verify that
   koha-tmpl/intranet-tmpl/prog/fr-FR/js/locale_data.js exists and
   contains your translation
6. Open your browser on the staff main page, change language and verify
   that the message is translated
7. Repeat 1-6 on OPAC side

Signed-off-by: Marc Véron <veron@veron.ch>
Comment 42 Marc Véron 2017-04-18 19:48:46 UTC
Created attachment 62329 [details] [review]
Bug 15395: Exclude xslt from the list of themes when building locale_data.js

Signed-off-by: Marc Véron <veron@veron.ch>
Comment 43 Marc Véron 2017-04-18 19:49:41 UTC
Created attachment 62330 [details] [review]
Bug 15395: Fix PO header reading in po2json

Since version 0.24 Locale::PO do not escape newline characters in string
returned by dequote()
This patch set the required version of Locale::PO to 0.24 and fix po2json

https://rt.cpan.org/Public/Bug/Display.html?id=96016
Signed-off-by: Marc Véron <veron@veron.ch>
Comment 44 Marc Véron 2017-04-18 19:50:38 UTC
Created attachment 62331 [details] [review]
Bug 15395: Fix output of msgcat

By default, msgcat tries to merge translations when two or more files have the
same msgid, even for the header (msgid ""). This resulted in an invalid PO file
("#-#-#-#-#" markers in header).
This patch tells msgcat to use the first available translation instead.

Also, this patch set correctly the Project-Id-Version field

Signed-off-by: Marc Véron <veron@veron.ch>
Comment 45 Marc Véron 2017-04-18 19:51:42 UTC
Created attachment 62332 [details] [review]
Bug 15395: Make Koha::I18N work with Plack + add tests

Language detection was done in a BEGIN block so it was not working correctly
with Plack. This is now done when the first translation is requested.

This patch also fixes a bug when set_locale(LC_ALL, ...) fails for some reasons
(locale is not installed system-wide for example).

And finally, add some unit tests

Test plan:
1. prove t/Koha/I18N.t
2. Replay the previous test plans with plack enabled

Signed-off-by: Marc Véron <veron@veron.ch>
Comment 46 Marc Véron 2017-04-18 19:53:01 UTC
Created attachment 62333 [details] [review]
Bug 15395: Search .js files in more directories

Specifically, koha-tmpl/intranet-tmpl/prog/js and
koha-tmpl/opac-tmpl/bootstrap/js

Signed-off-by: Marc Véron <veron@veron.ch>
Comment 47 Marc Véron 2017-04-18 19:57:19 UTC
Created attachment 62335 [details] [review]
Bug 15395: Example usage of I18N Template::Toolkit plugin

Moved this patch for example usage to teh end of patch set
and resolved minor conflict while applyin
Signed-off-by: Marc Véron <veron@veron.ch>
Comment 48 Julian Maurice 2017-04-19 07:36:26 UTC
Thanks for testing Marc!
Have you also tested translation with context (regarding bug 11285) ?
Comment 49 Marc Véron 2017-04-19 12:19:06 UTC
(In reply to Julian Maurice from comment #48)
> Thanks for testing Marc!
> Have you also tested translation with context (regarding bug 11285) ?

Just tested:
- Inserted the following in a template file:
<p>AAAA: [% tp("Biblio","Title") %]</p>
<p>BBBB: [% tp("Person","Title") %]</p>

- translate update de-DE

- Edit misc/translator/po/de_DE/LC_MESSAGES/Koha.po

#: koha-tmpl/intranet-tmpl/prog/en/modules/catalogue/results.tt:1
msgctxt "Biblio"
msgid "Title"
msgstr "Überschrift"

#: koha-tmpl/intranet-tmpl/prog/en/modules/catalogue/results.tt:2
msgctxt "Person"
msgid "Title"
msgstr "Anrede"

- translate install de-DE

Went to page, displys in English:

AAAA: Title
BBBB: Title

...and in German (de-DE):
AAAA: Überschrift
BBBB: Anrede

Works great!
Comment 50 Jonathan Druart 2017-04-25 19:23:12 UTC
[kohadev-koha@kohadevbox:/home/vagrant/kohaclone/misc/translator (bug_15395) ✭]% perl translate install fr-FR
/bin/cp: cannot stat ‘Koha.pot’: No such file or directory
/usr/bin/msgfmt: error while opening "/home/vagrant/kohaclone/misc/translator/po/fr_FR/LC_MESSAGES/Koha.po" for reading: No such file or directory
/home/vagrant/kohaclone/misc/translator/po2json {-p} {file.po} > {outputfile.json}
    -p  : do pretty-printing of json data


What did I miss?
Comment 51 Julian Maurice 2017-04-26 07:17:36 UTC
(In reply to Jonathan Druart from comment #50)
> What did I miss?

For now you need to run `translate update fr-FR` first to create the PO file, then you need to change the Plural-Forms header, and only after that you can run `translate install` to compile the PO into MO.

I think we can avoid the manual step of setting Plural-Forms by using msginit. I'll give it a try.
Comment 52 Julian Maurice 2017-04-26 09:07:24 UTC
Created attachment 62709 [details] [review]
Bug 15395: Fix several issues in translation process

- Change location of generated PO files to ease its integration into Pootle
  (po/xx-XX-messages.po instead of xx_XX/LC_MESSAGES/Koha.po)
- Use msginit for PO creation to try to set correctly Plural-Forms header. If it
  fails, set a default one
- 'install' command now creates the PO file if it doesn't already exist
- Fix locale name when language code is different from ll-CC (where ll is the
  language code and CC is the country code).

Test plan:
1. `perl translate install ar-Arab` should results in
   po/ar/LC_MESSAGES/Koha.mo and po/ar-Arab-messages.po
2. `perl translate install gl` should results in
   po/gl/LC_MESSAGES/Koha.mo and po/gl-messages.po
3. `perl translate install zh-Hans-CN` should results in
   po/zh_CN/LC_MESSAGES/Koha.mo and po/zh-Hans-CN-messages.po
4. `perl translate install fr-FR` should results in
   po/fr_FR/LC_MESSAGES/Koha.mo and po/fr-FR-messages.po
Comment 53 Marcel de Rooy 2017-07-14 08:45:55 UTC
This patchset includes a new perl dep: Locale::Util. You call set_locale two times. I am not sure if it is packaged for Debian; could not find it so quickly.
Should be added to PerlDependencies?

Also do we really need it? Can we use POSIX instead, or do without it, construct it some other way ?? Glancing through CPAN, I note: It is not the recommended way to set the program locale for a regular application.

Changing status to reflect need for feedback.
Comment 54 Mirko Tietgen 2017-07-14 08:58:32 UTC
Regarding packages, I found the following:

> $ apt-file search Locale::Util
> libdatetime-locale-perl: /usr/share/man/man3/DateTime::Locale::Util.3pm.gz
> libintl-perl: /usr/share/man/man3/Locale::Util.3pm.gz
> libsharyanto-utils-perl: /usr/share/man/man3/SHARYANTO::Locale::Util.3pm.gz
> 
> $ rmadison libintl-perl
> debian:
>  libintl-perl | 1.20-1        | squeeze                  | source, all
>  libintl-perl | 1.20-1        | wheezy                   | source, all
>  libintl-perl | 1.23-1        | jessie-kfreebsd          | source, all
>  libintl-perl | 1.23-1+deb8u1 | jessie-kfreebsd-security | source, all
>  libintl-perl | 1.23-1+deb8u1 | jessie-security          | source, all
>  libintl-perl | 1.23-1+deb8u1 | jessie                   | source, all
>  libintl-perl | 1.26-2        | stretch                  | source, all
>  libintl-perl | 1.26-2        | buster                   | source, all
>  libintl-perl | 1.26-2        | sid                      | source, all


If that is the correct package and the version in Jessie is ok, I don't see a problem.

This is only about availability of the dependency, not about the other questions Marcel brought up.
Comment 55 Julian Maurice 2017-07-24 16:11:52 UTC
(In reply to Marcel de Rooy from comment #53)
> Also do we really need it? Can we use POSIX instead, or do without it,
> construct it some other way ?? Glancing through CPAN, I note: It is not the
> recommended way to set the program locale for a regular application.

When I searched for the recommended way to set the program locale, it appeared to be with Locale::Util's set_locale. But after playing with this patchset a little, I noticed some small differences between using a locale that is installed system-wide and using a locale that is not installed (when set_locale returns false). So it might be better to not use set_locale or POSIX's setlocale, and just set the environment variables LANGUAGE, LANG and OUTPUT_CHARSET (never use the system locale, even if installed)
Comment 56 Julian Maurice 2017-08-11 10:44:27 UTC
Created attachment 65894 [details] [review]
Bug 15395: Remove the use of Locale::Util's set_locale