Bug 21156 - Internationalization: plural forms, context, and more for JS files
Summary: Internationalization: plural forms, context, and more for JS files
Status: Pushed to stable
Alias: None
Product: Koha
Classification: Unclassified
Component: I18N/L10N (show other bugs)
Version: master
Hardware: All All
: P5 - low enhancement (vote)
Assignee: Julian Maurice
QA Contact: Martin Renvoize
URL:
Keywords:
: 4503 (view as bug list)
Depends on: 15395 24734
Blocks: 15522 25317 25318 25319 25320 25321 25351 26065 26118 26217 26225 26226 26229 26230 26237 26240 26242 26243 26244 26256 26257 26261 26291 26334 26339 26344 26395 26439 26441 26979 24661 24725
  Show dependency treegraph
 
Reported: 2018-08-03 15:32 UTC by Julian Maurice
Modified: 2020-11-09 18:00 UTC (History)
9 users (show)

See Also:
Change sponsored?: ---
Patch complexity: ---
Text to go in the release notes:
Version(s) released in:
20.05.00, 19.11.04


Attachments
Bug 21156: Add plural translation capabilities to JS files (109.52 KB, patch)
2018-08-03 15:32 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 21156: Add plural translation capabilities to JS files (108.61 KB, patch)
2019-08-05 03:02 UTC, Jonathan Druart
Details | Diff | Splinter Review
Bug 21156: Add plural translation capabilities to JS files (108.78 KB, patch)
2020-01-14 12:07 UTC, Jonathan Druart
Details | Diff | Splinter Review
Bug 21156: Add plural translation capabilities to JS files (108.80 KB, patch)
2020-01-14 12:16 UTC, Martin Renvoize
Details | Diff | Splinter Review
Bug 21156: Add plural translation capabilities to JS files (108.98 KB, patch)
2020-02-06 14:22 UTC, Bernardo Gonzalez Kriegel
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Julian Maurice 2018-08-03 15:32:01 UTC

    
Comment 1 Julian Maurice 2018-08-03 15:32:52 UTC
Created attachment 77477 [details] [review]
Bug 21156: Add plural translation capabilities to JS files

It adds Javascript equivalent of Koha::I18N's exported subroutines, and
they are used the same way.

String extraction is done only on *.js files and require gettext 0.19
(available in Debian jessie, and also in wheezy-backports)

It adds Javascript library Gettext.js for handling translation and a
Perl script po2json to transform PO file into JSON.

Gettext.js and po2json both come from Locale::Simple.
There are several tools named po2json. It's simpler to integrate this
one into Koha than to check if the good one is installed on the system.
Locale::Simple is not needed.

To avoid polluting the global namespace too much, this patch also
introduce a global JS object named Koha and add some stuff in Koha.i18n

Test plan:
1. Add a translatable string in a JS file. For example, add this:
     alert(__nx("There is one item", "There are {count} items", 3,
     {count: 3}));
   to staff-global.js
2. cd misc/translator && ./translate update fr-FR
3. Open misc/translator/po/fr-FR-messages-js.po, verify that your
   string is present, and translate it
4. cd misc/translator && ./translate install fr-FR
5. (Optional) Verify that
   koha-tmpl/intranet-tmpl/prog/fr-FR/js/locale_data.js exists and
   contains your translation
6. Open your browser on the staff main page, change language and verify
   that the message is translated
7. Repeat 1-6 on OPAC side

Signed-off-by: Marc Véron <veron@veron.ch>
Comment 2 Katrin Fischer 2018-10-01 06:05:11 UTC
1. Add a translatable string in a JS file. For example, add this:
     alert(__nx("There is one item", "There are {count} items", 3,
     {count: 3}));

It looks like we already have a way of doing this with the format() syntax.

Example:
http://git.koha-community.org/gitweb/?p=koha.git;a=blob;f=koha-tmpl/intranet-tmpl/prog/en/includes/select2.inc;hb=705571a2b870fbb652ca210e32ee3e7ca050e75f#l45

How would plural translation work? Can you give examples for context and plurals?
Comment 3 Julian Maurice 2018-10-01 07:31:04 UTC
(In reply to Katrin Fischer from comment #2)
> 1. Add a translatable string in a JS file. For example, add this:
>      alert(__nx("There is one item", "There are {count} items", 3,
>      {count: 3}));
> 
> It looks like we already have a way of doing this with the format() syntax.
Variable substitution works the same way as format() but allows to give names to substitutions, to ease the work of translators (ex: "I like {food} and {drink}" is easier to read and translate than "I like %s and %s")

> How would plural translation work? Can you give examples for context and
> plurals?
For plural translation, gettext will select the correct string based on the integer parameter (the 3rd for __n and __nx, the 4th for __np and __npx) and the Plural-Forms header in the PO file
For instance,
  var count = <a positive integer>
  __n('item', 'items', count)

In English, __n will return 'item' if count == 1, 'items' otherwise
In French, __n will return 'exemplaire' if count == 0 or count == 1, 'exemplaires' otherwise, because Plural-Forms is "nplurals=2; plural=(n > 1);
In Slovenian, __n can return up to 4 different strings, because Slovenian has 4 different "plural forms"

The context is just another string to help disambiguation. Ex:
  __p('Bibliographic record', 'item')

This will appear in the po file as
  msgctxt "Bibliographic record"
  msgid "item"

And you can combine context and plural forms
  __np('Bibliographic record', 'item', 'items', count)
Comment 4 Jonathan Druart 2019-08-05 03:02:25 UTC
Created attachment 91976 [details] [review]
Bug 21156: Add plural translation capabilities to JS files

It adds Javascript equivalent of Koha::I18N's exported subroutines, and
they are used the same way.

String extraction is done only on *.js files and require gettext 0.19
(available in Debian jessie, and also in wheezy-backports)

It adds Javascript library Gettext.js for handling translation and a
Perl script po2json to transform PO file into JSON.

Gettext.js and po2json both come from Locale::Simple.
There are several tools named po2json. It's simpler to integrate this
one into Koha than to check if the good one is installed on the system.
Locale::Simple is not needed.

To avoid polluting the global namespace too much, this patch also
introduce a global JS object named Koha and add some stuff in Koha.i18n

Test plan:
1. Add a translatable string in a JS file. For example, add this:
     alert(__nx("There is one item", "There are {count} items", 3,
     {count: 3}));
   to staff-global.js
2. cd misc/translator && ./translate update fr-FR
3. Open misc/translator/po/fr-FR-messages-js.po, verify that your
   string is present, and translate it
4. cd misc/translator && ./translate install fr-FR
5. (Optional) Verify that
   koha-tmpl/intranet-tmpl/prog/fr-FR/js/locale_data.js exists and
   contains your translation
6. Open your browser on the staff main page, change language and verify
   that the message is translated
7. Repeat 1-6 on OPAC side
Comment 5 Jonathan Druart 2019-12-11 09:37:27 UTC
*** Bug 4503 has been marked as a duplicate of this bug. ***
Comment 6 Jonathan Druart 2020-01-14 12:07:46 UTC
Created attachment 97345 [details] [review]
Bug 21156: Add plural translation capabilities to JS files

It adds Javascript equivalent of Koha::I18N's exported subroutines, and
they are used the same way.

String extraction is done only on *.js files and require gettext 0.19
(available in Debian jessie, and also in wheezy-backports)

It adds Javascript library Gettext.js for handling translation and a
Perl script po2json to transform PO file into JSON.

Gettext.js and po2json both come from Locale::Simple.
There are several tools named po2json. It's simpler to integrate this
one into Koha than to check if the good one is installed on the system.
Locale::Simple is not needed.

To avoid polluting the global namespace too much, this patch also
introduce a global JS object named Koha and add some stuff in Koha.i18n

Test plan:
1. Add a translatable string in a JS file. For example, add this:
     alert(__nx("There is one item", "There are {count} items", 3,
     {count: 3}));
   to staff-global.js
2. cd misc/translator && ./translate update fr-FR
3. Open misc/translator/po/fr-FR-messages-js.po, verify that your
   string is present, and translate it
4. cd misc/translator && ./translate install fr-FR
5. (Optional) Verify that
   koha-tmpl/intranet-tmpl/prog/fr-FR/js/locale_data.js exists and
   contains your translation
6. Open your browser on the staff main page, change language and verify
   that the message is translated
7. Repeat 1-6 on OPAC side
Comment 7 Jonathan Druart 2020-01-14 12:08:46 UTC
Last rebase (from comment 4) contains a wrong conflict resolution.
Comment 8 Martin Renvoize 2020-01-14 12:16:38 UTC
Created attachment 97346 [details] [review]
Bug 21156: Add plural translation capabilities to JS files

It adds Javascript equivalent of Koha::I18N's exported subroutines, and
they are used the same way.

String extraction is done only on *.js files and require gettext 0.19
(available in Debian jessie, and also in wheezy-backports)

It adds Javascript library Gettext.js for handling translation and a
Perl script po2json to transform PO file into JSON.

Gettext.js and po2json both come from Locale::Simple.
There are several tools named po2json. It's simpler to integrate this
one into Koha than to check if the good one is installed on the system.
Locale::Simple is not needed.

To avoid polluting the global namespace too much, this patch also
introduce a global JS object named Koha and add some stuff in Koha.i18n

Test plan:
1. Add a translatable string in a JS file. For example, add this:
     alert(__nx("There is one item", "There are {count} items", 3,
     {count: 3}));
   to staff-global.js
2. cd misc/translator && ./translate update fr-FR
3. Open misc/translator/po/fr-FR-messages-js.po, verify that your
   string is present, and translate it
4. cd misc/translator && ./translate install fr-FR
5. (Optional) Verify that
   koha-tmpl/intranet-tmpl/prog/fr-FR/js/locale_data.js exists and
   contains your translation
6. Open your browser on the staff main page, change language and verify
   that the message is translated
7. Repeat 1-6 on OPAC side

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 9 Martin Renvoize 2020-01-14 12:17:24 UTC
Looks good, and passes a few quick tests.. I'm going to give it an SO stamp, but would love to see someone more familiar with the translations process give it a once over. :)
Comment 10 Bernardo Gonzalez Kriegel 2020-02-06 14:09:44 UTC
This works well, but qa-tool shows an error:

Processing files before patches
|========================>| 7 / 7 (100.00%)
Processing files after patches
|===>                     | 1 / 7 (14.29%)

An error occurred : Unmatched [ in regex; marked by <-- HERE in m/_\(\(?<string>[ <-- HERE ^\]*)\\)/ at /.../qa-test-tools/QohA/File/JS.pm line 29.

Is this a false positive?
Comment 11 Jonathan Druart 2020-02-06 14:11:54 UTC
(In reply to Bernardo Gonzalez Kriegel from comment #10)
> This works well, but qa-tool shows an error:
> 
> Processing files before patches
> |========================>| 7 / 7 (100.00%)
> Processing files after patches
> |===>                     | 1 / 7 (14.29%)
> 
> An error occurred : Unmatched [ in regex; marked by <-- HERE in
> m/_\(\(?<string>[ <-- HERE ^\]*)\\)/ at /.../qa-test-tools/QohA/File/JS.pm
> line 29.
> 
> Is this a false positive?

Yes, definitely, I am going to open an issue.
Comment 12 Jonathan Druart 2020-02-06 14:15:03 UTC
(In reply to Jonathan Druart from comment #11)
> (In reply to Bernardo Gonzalez Kriegel from comment #10)
> > This works well, but qa-tool shows an error:
> > 
> > Processing files before patches
> > |========================>| 7 / 7 (100.00%)
> > Processing files after patches
> > |===>                     | 1 / 7 (14.29%)
> > 
> > An error occurred : Unmatched [ in regex; marked by <-- HERE in
> > m/_\(\(?<string>[ <-- HERE ^\]*)\\)/ at /.../qa-test-tools/QohA/File/JS.pm
> > line 29.
> > 
> > Is this a false positive?
> 
> Yes, definitely, I am going to open an issue.

https://gitlab.com/koha-community/qa-test-tools/issues/23
Comment 13 Bernardo Gonzalez Kriegel 2020-02-06 14:22:50 UTC
Created attachment 98535 [details] [review]
Bug 21156: Add plural translation capabilities to JS files

It adds Javascript equivalent of Koha::I18N's exported subroutines, and
they are used the same way.

String extraction is done only on *.js files and require gettext 0.19
(available in Debian jessie, and also in wheezy-backports)

It adds Javascript library Gettext.js for handling translation and a
Perl script po2json to transform PO file into JSON.

Gettext.js and po2json both come from Locale::Simple.
There are several tools named po2json. It's simpler to integrate this
one into Koha than to check if the good one is installed on the system.
Locale::Simple is not needed.

To avoid polluting the global namespace too much, this patch also
introduce a global JS object named Koha and add some stuff in Koha.i18n

Test plan:
1. Add a translatable string in a JS file. For example, add this:
     alert(__nx("There is one item", "There are {count} items", 3,
     {count: 3}));
   to staff-global.js
2. cd misc/translator && ./translate update fr-FR
3. Open misc/translator/po/fr-FR-messages-js.po, verify that your
   string is present, and translate it
4. cd misc/translator && ./translate install fr-FR
5. (Optional) Verify that
   koha-tmpl/intranet-tmpl/prog/fr-FR/js/locale_data.js exists and
   contains your translation
6. Open your browser on the staff main page, change language and verify
   that the message is translated
7. Repeat 1-6 on OPAC side

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Works well, translation is Ok and test message is displayed correctly.
Current qa-tool error is a false positive.
Comment 14 Bernardo Gonzalez Kriegel 2020-02-06 14:35:16 UTC
(In reply to Jonathan Druart from comment #12)
...
> > Yes, definitely, I am going to open an issue.
> 
> https://gitlab.com/koha-community/qa-test-tools/issues/23

Good!
Comment 15 Martin Renvoize 2020-02-06 14:42:59 UTC
I'm going to take that second signoff from Bernardo as a PQA as the resident translation expert.  I found no regressions in my own testing and am confident in the change..

Thanks everyone!
Comment 16 Martin Renvoize 2020-02-10 10:17:18 UTC
Nice work everyone!

Pushed to master for 20.05
Comment 17 Joy Nelson 2020-03-05 00:42:48 UTC
enhancement not pushed to 19.11.x branch
Comment 18 Joy Nelson 2020-03-05 23:42:10 UTC
Correction-
Pushed to 19.11.x branch for 19.11.04

Thanks martin!