Bug 21156

Summary: Internationalization: plural forms, context, and more for JS files
Product: Koha Reporter: Julian Maurice <julian.maurice>
Component: I18N/L10NAssignee: Julian Maurice <julian.maurice>
Status: CLOSED FIXED QA Contact: Martin Renvoize <martin.renvoize>
Severity: enhancement    
Priority: P5 - low CC: 1joynelson, bgkriegel, black23, f.demians, jonathan.druart, josef.moravec, katrin.fischer, martin.renvoize, rbit
Version: Main   
Hardware: All   
OS: All   
See Also: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22490
Change sponsored?: --- Patch complexity: ---
Documentation contact: Documentation submission:
Text to go in the release notes:
Version(s) released in:
20.05.00, 19.11.04
Bug Depends on: 15395, 24734    
Bug Blocks: 15522, 25318, 25319, 26344, 30445, 34038, 24661, 24725, 25317, 25320, 25321, 25351, 26065, 26118, 26217, 26225, 26226, 26229, 26230, 26237, 26240, 26242, 26243, 26244, 26256, 26257, 26261, 26291, 26334, 26339, 26395, 26439, 26441, 26979, 28536, 33983, 34026, 34031, 34034, 34035, 34043    
Attachments: Bug 21156: Add plural translation capabilities to JS files
Bug 21156: Add plural translation capabilities to JS files
Bug 21156: Add plural translation capabilities to JS files
Bug 21156: Add plural translation capabilities to JS files
Bug 21156: Add plural translation capabilities to JS files

Description Julian Maurice 2018-08-03 15:32:01 UTC

    
Comment 1 Julian Maurice 2018-08-03 15:32:52 UTC
Created attachment 77477 [details] [review]
Bug 21156: Add plural translation capabilities to JS files

It adds Javascript equivalent of Koha::I18N's exported subroutines, and
they are used the same way.

String extraction is done only on *.js files and require gettext 0.19
(available in Debian jessie, and also in wheezy-backports)

It adds Javascript library Gettext.js for handling translation and a
Perl script po2json to transform PO file into JSON.

Gettext.js and po2json both come from Locale::Simple.
There are several tools named po2json. It's simpler to integrate this
one into Koha than to check if the good one is installed on the system.
Locale::Simple is not needed.

To avoid polluting the global namespace too much, this patch also
introduce a global JS object named Koha and add some stuff in Koha.i18n

Test plan:
1. Add a translatable string in a JS file. For example, add this:
     alert(__nx("There is one item", "There are {count} items", 3,
     {count: 3}));
   to staff-global.js
2. cd misc/translator && ./translate update fr-FR
3. Open misc/translator/po/fr-FR-messages-js.po, verify that your
   string is present, and translate it
4. cd misc/translator && ./translate install fr-FR
5. (Optional) Verify that
   koha-tmpl/intranet-tmpl/prog/fr-FR/js/locale_data.js exists and
   contains your translation
6. Open your browser on the staff main page, change language and verify
   that the message is translated
7. Repeat 1-6 on OPAC side

Signed-off-by: Marc VĂ©ron <veron@veron.ch>
Comment 2 Katrin Fischer 2018-10-01 06:05:11 UTC
1. Add a translatable string in a JS file. For example, add this:
     alert(__nx("There is one item", "There are {count} items", 3,
     {count: 3}));

It looks like we already have a way of doing this with the format() syntax.

Example:
http://git.koha-community.org/gitweb/?p=koha.git;a=blob;f=koha-tmpl/intranet-tmpl/prog/en/includes/select2.inc;hb=705571a2b870fbb652ca210e32ee3e7ca050e75f#l45

How would plural translation work? Can you give examples for context and plurals?
Comment 3 Julian Maurice 2018-10-01 07:31:04 UTC
(In reply to Katrin Fischer from comment #2)
> 1. Add a translatable string in a JS file. For example, add this:
>      alert(__nx("There is one item", "There are {count} items", 3,
>      {count: 3}));
> 
> It looks like we already have a way of doing this with the format() syntax.
Variable substitution works the same way as format() but allows to give names to substitutions, to ease the work of translators (ex: "I like {food} and {drink}" is easier to read and translate than "I like %s and %s")

> How would plural translation work? Can you give examples for context and
> plurals?
For plural translation, gettext will select the correct string based on the integer parameter (the 3rd for __n and __nx, the 4th for __np and __npx) and the Plural-Forms header in the PO file
For instance,
  var count = <a positive integer>
  __n('item', 'items', count)

In English, __n will return 'item' if count == 1, 'items' otherwise
In French, __n will return 'exemplaire' if count == 0 or count == 1, 'exemplaires' otherwise, because Plural-Forms is "nplurals=2; plural=(n > 1);
In Slovenian, __n can return up to 4 different strings, because Slovenian has 4 different "plural forms"

The context is just another string to help disambiguation. Ex:
  __p('Bibliographic record', 'item')

This will appear in the po file as
  msgctxt "Bibliographic record"
  msgid "item"

And you can combine context and plural forms
  __np('Bibliographic record', 'item', 'items', count)
Comment 4 Jonathan Druart 2019-08-05 03:02:25 UTC
Created attachment 91976 [details] [review]
Bug 21156: Add plural translation capabilities to JS files

It adds Javascript equivalent of Koha::I18N's exported subroutines, and
they are used the same way.

String extraction is done only on *.js files and require gettext 0.19
(available in Debian jessie, and also in wheezy-backports)

It adds Javascript library Gettext.js for handling translation and a
Perl script po2json to transform PO file into JSON.

Gettext.js and po2json both come from Locale::Simple.
There are several tools named po2json. It's simpler to integrate this
one into Koha than to check if the good one is installed on the system.
Locale::Simple is not needed.

To avoid polluting the global namespace too much, this patch also
introduce a global JS object named Koha and add some stuff in Koha.i18n

Test plan:
1. Add a translatable string in a JS file. For example, add this:
     alert(__nx("There is one item", "There are {count} items", 3,
     {count: 3}));
   to staff-global.js
2. cd misc/translator && ./translate update fr-FR
3. Open misc/translator/po/fr-FR-messages-js.po, verify that your
   string is present, and translate it
4. cd misc/translator && ./translate install fr-FR
5. (Optional) Verify that
   koha-tmpl/intranet-tmpl/prog/fr-FR/js/locale_data.js exists and
   contains your translation
6. Open your browser on the staff main page, change language and verify
   that the message is translated
7. Repeat 1-6 on OPAC side
Comment 5 Jonathan Druart 2019-12-11 09:37:27 UTC
*** Bug 4503 has been marked as a duplicate of this bug. ***
Comment 6 Jonathan Druart 2020-01-14 12:07:46 UTC
Created attachment 97345 [details] [review]
Bug 21156: Add plural translation capabilities to JS files

It adds Javascript equivalent of Koha::I18N's exported subroutines, and
they are used the same way.

String extraction is done only on *.js files and require gettext 0.19
(available in Debian jessie, and also in wheezy-backports)

It adds Javascript library Gettext.js for handling translation and a
Perl script po2json to transform PO file into JSON.

Gettext.js and po2json both come from Locale::Simple.
There are several tools named po2json. It's simpler to integrate this
one into Koha than to check if the good one is installed on the system.
Locale::Simple is not needed.

To avoid polluting the global namespace too much, this patch also
introduce a global JS object named Koha and add some stuff in Koha.i18n

Test plan:
1. Add a translatable string in a JS file. For example, add this:
     alert(__nx("There is one item", "There are {count} items", 3,
     {count: 3}));
   to staff-global.js
2. cd misc/translator && ./translate update fr-FR
3. Open misc/translator/po/fr-FR-messages-js.po, verify that your
   string is present, and translate it
4. cd misc/translator && ./translate install fr-FR
5. (Optional) Verify that
   koha-tmpl/intranet-tmpl/prog/fr-FR/js/locale_data.js exists and
   contains your translation
6. Open your browser on the staff main page, change language and verify
   that the message is translated
7. Repeat 1-6 on OPAC side
Comment 7 Jonathan Druart 2020-01-14 12:08:46 UTC
Last rebase (from comment 4) contains a wrong conflict resolution.
Comment 8 Martin Renvoize 2020-01-14 12:16:38 UTC
Created attachment 97346 [details] [review]
Bug 21156: Add plural translation capabilities to JS files

It adds Javascript equivalent of Koha::I18N's exported subroutines, and
they are used the same way.

String extraction is done only on *.js files and require gettext 0.19
(available in Debian jessie, and also in wheezy-backports)

It adds Javascript library Gettext.js for handling translation and a
Perl script po2json to transform PO file into JSON.

Gettext.js and po2json both come from Locale::Simple.
There are several tools named po2json. It's simpler to integrate this
one into Koha than to check if the good one is installed on the system.
Locale::Simple is not needed.

To avoid polluting the global namespace too much, this patch also
introduce a global JS object named Koha and add some stuff in Koha.i18n

Test plan:
1. Add a translatable string in a JS file. For example, add this:
     alert(__nx("There is one item", "There are {count} items", 3,
     {count: 3}));
   to staff-global.js
2. cd misc/translator && ./translate update fr-FR
3. Open misc/translator/po/fr-FR-messages-js.po, verify that your
   string is present, and translate it
4. cd misc/translator && ./translate install fr-FR
5. (Optional) Verify that
   koha-tmpl/intranet-tmpl/prog/fr-FR/js/locale_data.js exists and
   contains your translation
6. Open your browser on the staff main page, change language and verify
   that the message is translated
7. Repeat 1-6 on OPAC side

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 9 Martin Renvoize 2020-01-14 12:17:24 UTC
Looks good, and passes a few quick tests.. I'm going to give it an SO stamp, but would love to see someone more familiar with the translations process give it a once over. :)
Comment 10 Bernardo Gonzalez Kriegel 2020-02-06 14:09:44 UTC
This works well, but qa-tool shows an error:

Processing files before patches
|========================>| 7 / 7 (100.00%)
Processing files after patches
|===>                     | 1 / 7 (14.29%)

An error occurred : Unmatched [ in regex; marked by <-- HERE in m/_\(\(?<string>[ <-- HERE ^\]*)\\)/ at /.../qa-test-tools/QohA/File/JS.pm line 29.

Is this a false positive?
Comment 11 Jonathan Druart 2020-02-06 14:11:54 UTC
(In reply to Bernardo Gonzalez Kriegel from comment #10)
> This works well, but qa-tool shows an error:
> 
> Processing files before patches
> |========================>| 7 / 7 (100.00%)
> Processing files after patches
> |===>                     | 1 / 7 (14.29%)
> 
> An error occurred : Unmatched [ in regex; marked by <-- HERE in
> m/_\(\(?<string>[ <-- HERE ^\]*)\\)/ at /.../qa-test-tools/QohA/File/JS.pm
> line 29.
> 
> Is this a false positive?

Yes, definitely, I am going to open an issue.
Comment 12 Jonathan Druart 2020-02-06 14:15:03 UTC
(In reply to Jonathan Druart from comment #11)
> (In reply to Bernardo Gonzalez Kriegel from comment #10)
> > This works well, but qa-tool shows an error:
> > 
> > Processing files before patches
> > |========================>| 7 / 7 (100.00%)
> > Processing files after patches
> > |===>                     | 1 / 7 (14.29%)
> > 
> > An error occurred : Unmatched [ in regex; marked by <-- HERE in
> > m/_\(\(?<string>[ <-- HERE ^\]*)\\)/ at /.../qa-test-tools/QohA/File/JS.pm
> > line 29.
> > 
> > Is this a false positive?
> 
> Yes, definitely, I am going to open an issue.

https://gitlab.com/koha-community/qa-test-tools/issues/23
Comment 13 Bernardo Gonzalez Kriegel 2020-02-06 14:22:50 UTC
Created attachment 98535 [details] [review]
Bug 21156: Add plural translation capabilities to JS files

It adds Javascript equivalent of Koha::I18N's exported subroutines, and
they are used the same way.

String extraction is done only on *.js files and require gettext 0.19
(available in Debian jessie, and also in wheezy-backports)

It adds Javascript library Gettext.js for handling translation and a
Perl script po2json to transform PO file into JSON.

Gettext.js and po2json both come from Locale::Simple.
There are several tools named po2json. It's simpler to integrate this
one into Koha than to check if the good one is installed on the system.
Locale::Simple is not needed.

To avoid polluting the global namespace too much, this patch also
introduce a global JS object named Koha and add some stuff in Koha.i18n

Test plan:
1. Add a translatable string in a JS file. For example, add this:
     alert(__nx("There is one item", "There are {count} items", 3,
     {count: 3}));
   to staff-global.js
2. cd misc/translator && ./translate update fr-FR
3. Open misc/translator/po/fr-FR-messages-js.po, verify that your
   string is present, and translate it
4. cd misc/translator && ./translate install fr-FR
5. (Optional) Verify that
   koha-tmpl/intranet-tmpl/prog/fr-FR/js/locale_data.js exists and
   contains your translation
6. Open your browser on the staff main page, change language and verify
   that the message is translated
7. Repeat 1-6 on OPAC side

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Works well, translation is Ok and test message is displayed correctly.
Current qa-tool error is a false positive.
Comment 14 Bernardo Gonzalez Kriegel 2020-02-06 14:35:16 UTC
(In reply to Jonathan Druart from comment #12)
...
> > Yes, definitely, I am going to open an issue.
> 
> https://gitlab.com/koha-community/qa-test-tools/issues/23

Good!
Comment 15 Martin Renvoize 2020-02-06 14:42:59 UTC
I'm going to take that second signoff from Bernardo as a PQA as the resident translation expert.  I found no regressions in my own testing and am confident in the change..

Thanks everyone!
Comment 16 Martin Renvoize 2020-02-10 10:17:18 UTC
Nice work everyone!

Pushed to master for 20.05
Comment 17 Joy Nelson 2020-03-05 00:42:48 UTC
enhancement not pushed to 19.11.x branch
Comment 18 Joy Nelson 2020-03-05 23:42:10 UTC
Correction-
Pushed to 19.11.x branch for 19.11.04

Thanks martin!