Bug 20988 - Internationalization: wrap all translatable text inside t() calls
Summary: Internationalization: wrap all translatable text inside t() calls
Status: Needs Signoff
Alias: None
Product: Koha
Classification: Unclassified
Component: I18N/L10N (show other bugs)
Version: master
Hardware: All All
: P5 - low enhancement (vote)
Assignee: Julian Maurice
QA Contact: Testopia
URL:
Whiteboard:
Keywords:
Depends on: 15395
Blocks:
  Show dependency treegraph
 
Reported: 2018-06-25 07:17 UTC by Julian Maurice
Modified: 2018-11-16 15:15 UTC (History)
4 users (show)

See Also:
Change sponsored?: ---
Patch complexity: ---
Bot Control: ---
When did the bot last check this:
Who signed the patch off:
Text to go in the release notes:


Attachments
Bug 20988: Modify templates so that they can be parsed as HTML (31.48 KB, patch)
2018-06-25 07:20 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 20988: Allow to create po for all languages at once (1.19 KB, patch)
2018-06-25 07:20 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 20988: koha-tmpl/intranet-tmpl/prog/en/includes (478.66 KB, patch)
2018-06-25 07:21 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 20988: koha-tmpl/intranet-tmpl/prog/en/modules/help (901.31 KB, patch)
2018-06-25 07:21 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 20988: koha-tmpl/intranet-tmpl/prog/en/modules/cataloguing/value_builder/unimarc (895.53 KB, patch)
2018-06-25 07:21 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 20988: koha-tmpl/intranet-tmpl/prog/en/modules/cataloguing/value_builder (150.14 KB, patch)
2018-06-25 07:21 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 20988: koha-tmpl/intranet-tmpl/prog/en/modules/acqui (408.89 KB, patch)
2018-06-25 07:21 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 20988: koha-tmpl/intranet-tmpl/prog/en/modules/admin (710.14 KB, patch)
2018-06-25 07:21 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 20988: koha-tmpl/intranet-tmpl/prog/en/modules/authorities (44.30 KB, patch)
2018-06-25 07:22 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 20988: koha-tmpl/intranet-tmpl/prog/en/modules/basket (25.73 KB, patch)
2018-06-25 07:22 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 20988: koha-tmpl/intranet-tmpl/prog/en/modules/catalogue (178.82 KB, patch)
2018-06-25 07:22 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 20988: koha-tmpl/intranet-tmpl/prog/en/modules/cataloguing (117.23 KB, patch)
2018-06-25 07:22 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 20988: koha-tmpl/intranet-tmpl/prog/en/modules/circ (283.18 KB, patch)
2018-06-25 07:22 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 20988: koha-tmpl/intranet-tmpl/prog/en/modules/clubs (42.87 KB, patch)
2018-06-25 07:22 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 20988: koha-tmpl/intranet-tmpl/prog/en/modules/course_reserves (39.25 KB, patch)
2018-06-25 07:22 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 20988: koha-tmpl/intranet-tmpl/prog/en/modules/labels (95.21 KB, patch)
2018-06-25 07:22 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 20988: koha-tmpl/intranet-tmpl/prog/en/modules/members (311.54 KB, patch)
2018-06-25 07:22 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 20988: koha-tmpl/intranet-tmpl/prog/en/modules/onboarding (41.52 KB, patch)
2018-06-25 07:22 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 20988: koha-tmpl/intranet-tmpl/prog/en/modules/patron_lists (18.83 KB, patch)
2018-06-25 07:22 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 20988: koha-tmpl/intranet-tmpl/prog/en/modules/patroncards (93.53 KB, patch)
2018-06-25 07:23 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 20988: koha-tmpl/intranet-tmpl/prog/en/modules/plugins (17.93 KB, patch)
2018-06-25 07:23 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 20988: koha-tmpl/intranet-tmpl/prog/en/modules/installer (35.10 KB, patch)
2018-06-25 07:23 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 20988: koha-tmpl/intranet-tmpl/prog/en/modules/reports (246.49 KB, patch)
2018-06-25 07:23 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 20988: koha-tmpl/intranet-tmpl/prog/en/modules/rotating_collections (26.50 KB, patch)
2018-06-25 07:23 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 20988: koha-tmpl/intranet-tmpl/prog/en/modules/serials (233.71 KB, patch)
2018-06-25 07:23 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 20988: koha-tmpl/intranet-tmpl/prog/en/modules/tools (402.74 KB, patch)
2018-06-25 07:23 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 20988: koha-tmpl/intranet-tmpl/prog/en/modules/virtualshelves (46.06 KB, patch)
2018-06-25 07:23 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 20988: koha-tmpl/intranet-tmpl/prog/en/modules/offline_circ (13.80 KB, patch)
2018-06-25 07:23 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 20988: koha-tmpl/opac-tmpl/bootstrap/en/includes (132.59 KB, patch)
2018-06-25 07:23 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 20988: koha-tmpl/opac-tmpl/bootstrap/en/modules/opac (888.43 KB, patch)
2018-06-25 07:24 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 20988: koha-tmpl/opac-tmpl/bootstrap/en/modules (148.33 KB, patch)
2018-06-25 07:24 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 20988: koha-tmpl/intranet-tmpl/prog/en/modules (307.78 KB, patch)
2018-06-25 07:24 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 20988: Add a sample of a translated PO file (757.18 KB, patch)
2018-06-25 07:45 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 20988: Add template macros to ease template rewriting (3.02 KB, patch)
2018-07-13 08:29 UTC, Julian Maurice
Details | Diff | Splinter Review
Bug 20988: Add template macros to ease template rewriting (3.02 KB, patch)
2018-07-13 08:30 UTC, Julian Maurice
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Julian Maurice 2018-06-25 07:17:16 UTC
Bug 15395's Koha::I18n is working well, but it is basically useless until we wrap all the translatable strings inside __*() calls (or t*() calls inside templates).
This bug aims to do that.

I wrote a little tool to automatically find and replace translatable text by the corresponding call. It can be found here : https://github.com/jajm/koha-i18n.

This bug will hold a few patches that will make koha-i18n work better and I'll try to attach all the changes it does.
Comment 1 Julian Maurice 2018-06-25 07:20:26 UTC
Created attachment 76333 [details] [review]
Bug 20988: Modify templates so that they can be parsed as HTML

This is required for koha-i18n [1] to work correctly

[1] https://github.com/jajm/koha-i18n
Comment 2 Julian Maurice 2018-06-25 07:20:50 UTC
Created attachment 76334 [details] [review]
Bug 20988: Allow to create po for all languages at once

This is required for the script that comes with koha-i18n, which creates
the new PO files and fill them with existing translations.
Comment 3 Julian Maurice 2018-06-25 07:21:11 UTC
Created attachment 76335 [details] [review]
Bug 20988: koha-tmpl/intranet-tmpl/prog/en/includes
Comment 4 Julian Maurice 2018-06-25 07:21:23 UTC
Created attachment 76336 [details] [review]
Bug 20988: koha-tmpl/intranet-tmpl/prog/en/modules/help
Comment 5 Julian Maurice 2018-06-25 07:21:34 UTC
Created attachment 76337 [details] [review]
Bug 20988: koha-tmpl/intranet-tmpl/prog/en/modules/cataloguing/value_builder/unimarc
Comment 6 Julian Maurice 2018-06-25 07:21:40 UTC
Created attachment 76338 [details] [review]
Bug 20988: koha-tmpl/intranet-tmpl/prog/en/modules/cataloguing/value_builder
Comment 7 Julian Maurice 2018-06-25 07:21:48 UTC
Created attachment 76339 [details] [review]
Bug 20988: koha-tmpl/intranet-tmpl/prog/en/modules/acqui
Comment 8 Julian Maurice 2018-06-25 07:21:58 UTC
Created attachment 76340 [details] [review]
Bug 20988: koha-tmpl/intranet-tmpl/prog/en/modules/admin
Comment 9 Julian Maurice 2018-06-25 07:22:03 UTC
Created attachment 76341 [details] [review]
Bug 20988: koha-tmpl/intranet-tmpl/prog/en/modules/authorities
Comment 10 Julian Maurice 2018-06-25 07:22:08 UTC
Created attachment 76342 [details] [review]
Bug 20988: koha-tmpl/intranet-tmpl/prog/en/modules/basket
Comment 11 Julian Maurice 2018-06-25 07:22:14 UTC
Created attachment 76343 [details] [review]
Bug 20988: koha-tmpl/intranet-tmpl/prog/en/modules/catalogue
Comment 12 Julian Maurice 2018-06-25 07:22:20 UTC
Created attachment 76344 [details] [review]
Bug 20988: koha-tmpl/intranet-tmpl/prog/en/modules/cataloguing
Comment 13 Julian Maurice 2018-06-25 07:22:27 UTC
Created attachment 76345 [details] [review]
Bug 20988: koha-tmpl/intranet-tmpl/prog/en/modules/circ
Comment 14 Julian Maurice 2018-06-25 07:22:32 UTC
Created attachment 76346 [details] [review]
Bug 20988: koha-tmpl/intranet-tmpl/prog/en/modules/clubs
Comment 15 Julian Maurice 2018-06-25 07:22:36 UTC
Created attachment 76347 [details] [review]
Bug 20988: koha-tmpl/intranet-tmpl/prog/en/modules/course_reserves
Comment 16 Julian Maurice 2018-06-25 07:22:42 UTC
Created attachment 76348 [details] [review]
Bug 20988: koha-tmpl/intranet-tmpl/prog/en/modules/labels
Comment 17 Julian Maurice 2018-06-25 07:22:48 UTC
Created attachment 76349 [details] [review]
Bug 20988: koha-tmpl/intranet-tmpl/prog/en/modules/members
Comment 18 Julian Maurice 2018-06-25 07:22:52 UTC
Created attachment 76350 [details] [review]
Bug 20988: koha-tmpl/intranet-tmpl/prog/en/modules/onboarding
Comment 19 Julian Maurice 2018-06-25 07:22:58 UTC
Created attachment 76351 [details] [review]
Bug 20988: koha-tmpl/intranet-tmpl/prog/en/modules/patron_lists
Comment 20 Julian Maurice 2018-06-25 07:23:03 UTC
Created attachment 76352 [details] [review]
Bug 20988: koha-tmpl/intranet-tmpl/prog/en/modules/patroncards
Comment 21 Julian Maurice 2018-06-25 07:23:07 UTC
Created attachment 76353 [details] [review]
Bug 20988: koha-tmpl/intranet-tmpl/prog/en/modules/plugins
Comment 22 Julian Maurice 2018-06-25 07:23:12 UTC
Created attachment 76354 [details] [review]
Bug 20988: koha-tmpl/intranet-tmpl/prog/en/modules/installer
Comment 23 Julian Maurice 2018-06-25 07:23:18 UTC
Created attachment 76355 [details] [review]
Bug 20988: koha-tmpl/intranet-tmpl/prog/en/modules/reports
Comment 24 Julian Maurice 2018-06-25 07:23:23 UTC
Created attachment 76356 [details] [review]
Bug 20988: koha-tmpl/intranet-tmpl/prog/en/modules/rotating_collections
Comment 25 Julian Maurice 2018-06-25 07:23:31 UTC
Created attachment 76357 [details] [review]
Bug 20988: koha-tmpl/intranet-tmpl/prog/en/modules/serials
Comment 26 Julian Maurice 2018-06-25 07:23:39 UTC
Created attachment 76358 [details] [review]
Bug 20988: koha-tmpl/intranet-tmpl/prog/en/modules/tools
Comment 27 Julian Maurice 2018-06-25 07:23:43 UTC
Created attachment 76359 [details] [review]
Bug 20988: koha-tmpl/intranet-tmpl/prog/en/modules/virtualshelves
Comment 28 Julian Maurice 2018-06-25 07:23:48 UTC
Created attachment 76360 [details] [review]
Bug 20988: koha-tmpl/intranet-tmpl/prog/en/modules/offline_circ
Comment 29 Julian Maurice 2018-06-25 07:23:55 UTC
Created attachment 76361 [details] [review]
Bug 20988: koha-tmpl/opac-tmpl/bootstrap/en/includes
Comment 30 Julian Maurice 2018-06-25 07:24:07 UTC
Created attachment 76362 [details] [review]
Bug 20988: koha-tmpl/opac-tmpl/bootstrap/en/modules/opac
Comment 31 Julian Maurice 2018-06-25 07:24:14 UTC
Created attachment 76363 [details] [review]
Bug 20988: koha-tmpl/opac-tmpl/bootstrap/en/modules
Comment 32 Julian Maurice 2018-06-25 07:24:22 UTC
Created attachment 76364 [details] [review]
Bug 20988: koha-tmpl/intranet-tmpl/prog/en/modules
Comment 33 Julian Maurice 2018-06-25 07:45:24 UTC
Created attachment 76365 [details] [review]
Bug 20988: Add a sample of a translated PO file

PO files are too big to be attached on Bugzilla, so here's a gzipped
version of misc/translator/po/fr-FR-messages.po as generated by
create-messages.sh (found in https://github.com/jajm/koha-i18n)

After applying the patch, run
  gunzip misc/translator/po/fr-FR-messages.po.gz

A few numbers about this PO file:
- Number of messages: 14178
- Number of translated messages: 13631
- Number of non-fuzzy translated messages: 9720
Comment 34 Julian Maurice 2018-06-25 07:48:49 UTC
All theses patches (except the first two) were generated with the following commands:

  koha-i18n --in-place koha-tmpl/**/en/**/*.tt koha-tmpl/**/en/**/*.inc  # might 
be a zsh'ism, haven't tried with bash
  cd misc/translator && $GOPATH/src/github.com/jajm/koha-i18n/scripts/create-messages.sh
Comment 35 Julian Maurice 2018-06-25 07:54:46 UTC
Test plan:

1. Apply bug 15395
2. Apply the first two patches of this bug
3. Install koha-i18n (see the README)
4. Run koha-i18n on all .tt and .inc files
5. cd to misc/translator and run $GOPATH/src/github.com/jajm/koha-i18n/scripts/create-messages.sh
6. Run ./translate install fr-FR  # or any available language
7. Reload starman
8. Go to Koha with your browser, switch language to fr-FR and check that the strings translated in misc/translator/fr-FR-messages.po are translated in the interface.
Comment 36 paxed 2018-06-26 06:14:56 UTC
Unfortunately quite a lot of the messages are split in multiple parts, making translation very hard:

[% t('Warning: This file is version') %] [% message.upload_version %][% t(', but I only know how to import version') %] [% message.current_version %][% t('. I\'ll try my best.') %]


Currently they're much better, with placeholders:

"Warning: This file is version %s, but I only know how to import version %s. "
Comment 37 Julian Maurice 2018-06-26 06:52:59 UTC
(In reply to paxed from comment #36)
> Unfortunately quite a lot of the messages are split in multiple parts,
> making translation very hard:
> 
> [% t('Warning: This file is version') %] [% message.upload_version %][% t(',
> but I only know how to import version') %] [% message.current_version %][%
> t('. I\'ll try my best.') %]
> 
> 
> Currently they're much better, with placeholders:
> 
> "Warning: This file is version %s, but I only know how to import version %s.
> "

I agree. There's still some manual work, the goal of this bug is to reduce it to a minimum.

This particular example should be

[% tx('Warning: This file is version {upload_version} but I only know how to import {current_version}. I\'ll try my best.', { upload_version = upload_version, current_version = current_version) %]

I'm not sure we can accomplish that easily with an automated tool. It's probably easier to run the tool and fix manually the problematic ones.

But please note that, unlike actual PO files, the one attached doesn't contain strings like this:

"%s %s %s   %s %s - %s %s %s - %s%s %s- %s%s %s - %s%s %sEdition: %s%s "
"%s ; %s%s %s - %s%s %s ; %s%s %s "


Also, I attached all the changes to this one bug, but that doesn't mean we should do it all at once. We can take the safer way of changing one template file at a time (or one module at a time). It will certainly be easier to rebase/test/review.
Comment 38 paxed 2018-06-28 06:55:42 UTC
BTW, my take on reducing the number of TT placeholders is in bug 12221

(In reply to Julian Maurice from comment #37)
> 
> But please note that, unlike actual PO files, the one attached doesn't
> contain strings like this:
> 
> "%s %s %s   %s %s - %s %s %s - %s%s %s- %s%s %s - %s%s %sEdition: %s%s "
> "%s ; %s%s %s - %s%s %s ; %s%s %s "
>
Comment 39 Julian Maurice 2018-06-28 07:09:02 UTC
(In reply to paxed from comment #38)
> BTW, my take on reducing the number of TT placeholders is in bug 12221

I wasn't aware of this bug, thanks.
Comment 40 Julian Maurice 2018-07-13 08:29:36 UTC
Created attachment 76934 [details] [review]
Bug 20988: Add template macros to ease template rewriting
Comment 41 Julian Maurice 2018-07-13 08:30:13 UTC
Created attachment 76935 [details] [review]
Bug 20988: Add template macros to ease template rewriting
Comment 42 Julian Maurice 2018-07-13 08:32:29 UTC
There's a new version of koha-i18n, written in Perl this time. It is available at the same URL (https://github.com/jajm/koha-i18n).
It tries to wrap longer strings, so it should address comment 36.

I obsoleted all generated patches, the result can be seen here: https://gitlab.com/jajm/Koha/commits/bug/20988
Comment 43 Michal Denar 2018-09-13 12:04:23 UTC
Any testplan?
Comment 44 Julian Maurice 2018-09-13 12:41:33 UTC
(In reply to Michal Denar from comment #43)
> Any testplan?

Yes, see comment 35
Comment 45 Fridolin SOMERS 2018-09-24 14:48:27 UTC
Is it possible to use double-quotes ?
  [% t("translate me") %]
Because actually translations are made for text with double quotes especially for JavaScript. Because some translated text can add a single quote like :
today => aujourd'hui
Comment 46 Julian Maurice 2018-09-24 15:01:17 UTC
(In reply to Fridolin SOMERS from comment #45)
> Is it possible to use double-quotes ?
>   [% t("translate me") %]
> Because actually translations are made for text with double quotes
> especially for JavaScript. Because some translated text can add a single
> quote like :
> today => aujourd'hui
You can use single quotes or double quotes. It doesn't matter since strings are translated at runtime (and not replaced in the source file like `./translate install` does).

So you can write
[% t('A string with "double quotes"') %]
and
[% t("A string with 'single quotes'") %]
Comment 47 Fridolin SOMERS 2018-11-16 15:15:37 UTC
(In reply to Julian Maurice from comment #46)
> (In reply to Fridolin SOMERS from comment #45)
> > Is it possible to use double-quotes ?
> >   [% t("translate me") %]
> > Because actually translations are made for text with double quotes
> > especially for JavaScript. Because some translated text can add a single
> > quote like :
> > today => aujourd'hui
> You can use single quotes or double quotes. It doesn't matter since strings
> are translated at runtime (and not replaced in the source file like
> `./translate install` does).
> 
> So you can write
> [% t('A string with "double quotes"') %]
> and
> [% t("A string with 'single quotes'") %]

Ah of course. Nice ;)