Bug 35024

Summary: Do not wrap PO files
Product: Koha Reporter: Jonathan Druart <jonathan.druart>
Component: Architecture, internals, and plumbingAssignee: Jonathan Druart <jonathan.druart>
Status: CLOSED FIXED QA Contact: Martin Renvoize (ashimema) <martin.renvoize>
Severity: normal    
Priority: P5 - low CC: david, fridolin.somers, m.de.rooy, martin.renvoize
Version: unspecified   
Hardware: All   
OS: All   
See Also: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=34959
GIT URL: Change sponsored?: ---
Patch complexity: --- Documentation contact:
Documentation submission: Text to go in the release notes:
Version(s) released in:
23.11.00,23.05.06,22.11.12,22.05.17,21.11.26
Circulation function:
Bug Depends on:    
Bug Blocks: 35079    
Attachments: Bug 35024: Do not wrap translations
Bug 35024: Do not wrap translations
Bug 35024: Do not wrap translations

Description Jonathan Druart 2023-10-11 08:49:32 UTC
Looks like (thanks Julian!) it's coming from libunistring, between 0.9.10 and 1.0 I am seeing a change in how the strings in the PO files are wrapped.

Debian 11 (using libunistring 0.9):
"%s %sPrintemps%s %s %sÉté%s %s %sAutomne%s %s %sHiver%s %s %sPrtps%s %s %sÉté"
"%s %s %sAut%s %s %sHiv%s %s %s %s"

Debian 12, Ubuntu 22.04(using libunistring 1.0)
"%s %sPrintemps%s %s %sÉté%s %s %sAutomne%s %s %sHiver%s %s %sPrtps%s %s "
"%sÉté%s %s %sAut%s %s %sHiv%s %s %s %s"

It *could* come from this changes:
https://git.savannah.gnu.org/gitweb/?p=libunistring.git;a=blob;f=NEWS;h=5a43ddd7011d62a952733f6c0b7ad52aa4f385c7;hb=HEAD
   * The functions u*_possible_linebreaks and u*_width_linebreaks now make it
     easier to work with strings that contain CR-LF sequences: In this case,
     in the returned array, it will return UC_BREAK_CR_BEFORE_LF followed by
     UC_BREAK_MANDATORY (instead of twice UC_BREAK_MANDATORY).

The command used is:
% msgmerge --backup=off --quiet -F --update misc/translator/po/fr-FR-staff-prog.po misc/translator/Koha-staff-prog.pot

No matter the value of --width.
Comment 1 Jonathan Druart 2023-10-11 08:59:16 UTC
One solution is to use --no-wrap, and never wrap :)

I sent an email to the list about this suggestion https://lists.katipo.co.nz/pipermail/koha/2023-October/060143.html
Comment 2 Jonathan Druart 2023-10-17 11:48:03 UTC
Created attachment 157229 [details] [review]
Bug 35024: Do not wrap translations

PO files wrap differently depending on versions.

Looks like (thanks Julian!) it's coming from libunistring, between 0.9.10 and 1.0 I am seeing a change in how the strings in the PO files are wrapped.

Debian 11 (using libunistring 0.9):
"%s %sPrintemps%s %s %sÉté%s %s %sAutomne%s %s %sHiver%s %s %sPrtps%s %s %sÉté"
"%s %s %sAut%s %s %sHiv%s %s %s %s"

Debian 12, Ubuntu 22.04(using libunistring 1.0)
"%s %sPrintemps%s %s %sÉté%s %s %sAutomne%s %s %sHiver%s %s %sPrtps%s %s "
"%sÉté%s %s %sAut%s %s %sHiv%s %s %s %s"

It *could* come from this changes:
https://git.savannah.gnu.org/gitweb/?p=libunistring.git;a=blob;f=NEWS;h=5a43ddd7011d62a952733f6c0b7ad52aa4f385c7;hb=HEAD
   * The functions u*_possible_linebreaks and u*_width_linebreaks now make it
     easier to work with strings that contain CR-LF sequences: In this case,
     in the returned array, it will return UC_BREAK_CR_BEFORE_LF followed by
     UC_BREAK_MANDATORY (instead of twice UC_BREAK_MANDATORY).

The command used is:
% msgmerge --backup=off --quiet -F --update misc/translator/po/fr-FR-staff-prog.po misc/translator/Koha-staff-prog.pot

No matter the value of --width.

One solution is to use --no-wrap, and never wrap :)

I sent an email to the list about this suggestion https://lists.katipo.co.nz/pipermail/koha/2023-October/060143.html

Test plan:
% gulp po:update --lang LANG

And notice that the lines are no wrapped anymore.

Note that in master there were already not wrapped (?)
commit edd378f4d14d79107d662b053c8e6f750326dd23
Translation updates for Koha 23.05.00
Comment 3 Jonathan Druart 2023-10-17 11:52:18 UTC Comment hidden (obsolete)
Comment 4 Jonathan Druart 2023-10-17 11:58:43 UTC
Files will no longer be wrapped.
Comment 5 David Nind 2023-10-19 05:40:41 UTC
Created attachment 157388 [details] [review]
Bug 35024: Do not wrap translations

PO files wrap differently depending on versions.

Looks like (thanks Julian!) it's coming from libunistring, between 0.9.10 and 1.0 I am seeing a change in how the strings in the PO files are wrapped.

Debian 11 (using libunistring 0.9):
"%s %sPrintemps%s %s %sÉté%s %s %sAutomne%s %s %sHiver%s %s %sPrtps%s %s %sÉté"
"%s %s %sAut%s %s %sHiv%s %s %s %s"

Debian 12, Ubuntu 22.04(using libunistring 1.0)
"%s %sPrintemps%s %s %sÉté%s %s %sAutomne%s %s %sHiver%s %s %sPrtps%s %s "
"%sÉté%s %s %sAut%s %s %sHiv%s %s %s %s"

It *could* come from this changes:
https://git.savannah.gnu.org/gitweb/?p=libunistring.git;a=blob;f=NEWS;h=5a43ddd7011d62a952733f6c0b7ad52aa4f385c7;hb=HEAD
   * The functions u*_possible_linebreaks and u*_width_linebreaks now make it
     easier to work with strings that contain CR-LF sequences: In this case,
     in the returned array, it will return UC_BREAK_CR_BEFORE_LF followed by
     UC_BREAK_MANDATORY (instead of twice UC_BREAK_MANDATORY).

The command used is:
% msgmerge --backup=off --quiet -F --update misc/translator/po/fr-FR-staff-prog.po misc/translator/Koha-staff-prog.pot

No matter the value of --width.

One solution is to use --no-wrap, and never wrap :)

I sent an email to the list about this suggestion https://lists.katipo.co.nz/pipermail/koha/2023-October/060143.html

Test plan:
% gulp po:update --lang LANG

And notice that the lines are no wrapped anymore.

Note that in master there were already not wrapped (?)
commit edd378f4d14d79107d662b053c8e6f750326dd23
Translation updates for Koha 23.05.00

Signed-off-by: David Nind <david@davidnind.com>
Comment 6 David Nind 2023-10-19 05:54:59 UTC
Testing notes using KTD:

1. Applied patch.

2. Ran: gulp po:update --lang dr-FR

3. Searched misc/translator/po/fr-FR-staff-prog.po for Printemps - note that msgstr is now all on one line.
Comment 7 Martin Renvoize (ashimema) 2023-10-19 08:56:51 UTC
Created attachment 157399 [details] [review]
Bug 35024: Do not wrap translations

PO files wrap differently depending on versions.

Looks like (thanks Julian!) it's coming from libunistring, between 0.9.10 and 1.0 I am seeing a change in how the strings in the PO files are wrapped.

Debian 11 (using libunistring 0.9):
"%s %sPrintemps%s %s %sÉté%s %s %sAutomne%s %s %sHiver%s %s %sPrtps%s %s %sÉté"
"%s %s %sAut%s %s %sHiv%s %s %s %s"

Debian 12, Ubuntu 22.04(using libunistring 1.0)
"%s %sPrintemps%s %s %sÉté%s %s %sAutomne%s %s %sHiver%s %s %sPrtps%s %s "
"%sÉté%s %s %sAut%s %s %sHiv%s %s %s %s"

It *could* come from this changes:
https://git.savannah.gnu.org/gitweb/?p=libunistring.git;a=blob;f=NEWS;h=5a43ddd7011d62a952733f6c0b7ad52aa4f385c7;hb=HEAD
   * The functions u*_possible_linebreaks and u*_width_linebreaks now make it
     easier to work with strings that contain CR-LF sequences: In this case,
     in the returned array, it will return UC_BREAK_CR_BEFORE_LF followed by
     UC_BREAK_MANDATORY (instead of twice UC_BREAK_MANDATORY).

The command used is:
% msgmerge --backup=off --quiet -F --update misc/translator/po/fr-FR-staff-prog.po misc/translator/Koha-staff-prog.pot

No matter the value of --width.

One solution is to use --no-wrap, and never wrap :)

I sent an email to the list about this suggestion https://lists.katipo.co.nz/pipermail/koha/2023-October/060143.html

Test plan:
% gulp po:update --lang LANG

And notice that the lines are no wrapped anymore.

Note that in master there were already not wrapped (?)
commit edd378f4d14d79107d662b053c8e6f750326dd23
Translation updates for Koha 23.05.00

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 8 Martin Renvoize (ashimema) 2023-10-19 08:57:17 UTC
Trivial change, great improvement.

PO's look good and work as expected.

Passing QA
Comment 9 Tomás Cohen Arazi (tcohen) 2023-10-19 19:02:21 UTC
Pushed to master for 23.11.

Nice work everyone, thanks!
Comment 10 Fridolin Somers 2023-10-25 08:12:10 UTC
Pushed to 23.05.x for 23.05.05
Comment 11 Fridolin Somers 2023-11-03 20:28:38 UTC
Ah like Bug 34959 this is for next release.

Reverted