Bug 34959 - Translator tool generates too many changes
Summary: Translator tool generates too many changes
Status: RESOLVED FIXED
Alias: None
Product: Koha
Classification: Unclassified
Component: Architecture, internals, and plumbing (show other bugs)
Version: unspecified
Hardware: All All
: P5 - low major
Assignee: Jonathan Druart
QA Contact: Kyle M Hall (khall)
URL:
Keywords:
: 24797 (view as bug list)
Depends on:
Blocks: 35043
  Show dependency treegraph
 
Reported: 2023-09-29 13:09 UTC by Jonathan Druart
Modified: 2024-04-17 09:13 UTC (History)
8 users (show)

See Also:
Change sponsored?: ---
Patch complexity: ---
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:
23.11.00,23.05.06,22.11.12,22.05.17,21.11.26
Circulation function:


Attachments
Bug 34959: Remove unused sub (1.04 KB, patch)
2023-10-10 07:48 UTC, Jonathan Druart
Details | Diff | Splinter Review
Bug 34959: Sort PO files correctly (9.90 KB, patch)
2023-10-10 07:48 UTC, Jonathan Druart
Details | Diff | Splinter Review
Bug 34959: Remove unused sub (1.09 KB, patch)
2023-10-10 19:16 UTC, David Nind
Details | Diff | Splinter Review
Bug 34959: Sort PO files correctly (9.94 KB, patch)
2023-10-10 19:16 UTC, David Nind
Details | Diff | Splinter Review
Bug 34959: Remove unused sub (1.15 KB, patch)
2023-10-13 13:14 UTC, Kyle M Hall (khall)
Details | Diff | Splinter Review
Bug 34959: Sort PO files correctly (10.00 KB, patch)
2023-10-13 13:14 UTC, Kyle M Hall (khall)
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Jonathan Druart 2023-09-29 13:09:59 UTC
If you update the po files using `translate update es-ES`, commit your changes, then run the script again, you will notice that changes are still generated.

It's coming from the commented lines showing the #:filepath:line_number info

They should be ordered the same way for each run to prevent unnecessary changes.
Comment 1 Jonathan Druart 2023-10-10 07:48:43 UTC
Created attachment 156781 [details] [review]
Bug 34959: Remove unused sub

Nothing special here, this sub is not used and we can remove it.
Comment 2 Jonathan Druart 2023-10-10 07:48:45 UTC
Created attachment 156782 [details] [review]
Bug 34959: Sort PO files correctly

The PO files are not sorted when we update them which leads to
unnecessary changes that are commited: hard to see differences and
make git index grow superfluously.

Test plan:
0. Do not apply this patch
1. gulp po:update --lang es-ES
2. git commit -a -m"First PO update"
3. Run again the gulp update command
4. git diff
=> You have a lot of changes generated here, the po:update is not
idempotent.

5. Apply this patch
6. Run the gulp update command
7 git commit -a -m"PO update after 34959"
8. Run the gulp update command
9. git diff
=> No changes are generated

Note that this patch will all the entries by files, and per line
numbers.
It fixes a bug in some condition, where we add information/context about
the string. For instance search for "For the first occurrence" in the
file. Prior to this patch this was not correct, we didn't add info about
the first occurrence (but whichever in the list).
Comment 3 Jonathan Druart 2023-10-10 07:51:12 UTC
This is in the context of the migration to weblate
https://wiki.koha-community.org/wiki/Translation_server_migration_weblate

And the suggestion to let weblate deals with the PO files in their own git repository.
Comment 4 David Nind 2023-10-10 19:16:15 UTC
Created attachment 156806 [details] [review]
Bug 34959: Remove unused sub

Nothing special here, this sub is not used and we can remove it.

Signed-off-by: David Nind <david@davidnind.com>
Comment 5 David Nind 2023-10-10 19:16:18 UTC
Created attachment 156807 [details] [review]
Bug 34959: Sort PO files correctly

The PO files are not sorted when we update them which leads to
unnecessary changes that are commited: hard to see differences and
make git index grow superfluously.

Test plan:
0. Do not apply this patch
1. gulp po:update --lang es-ES
2. git commit -a -m"First PO update"
3. Run again the gulp update command
4. git diff
=> You have a lot of changes generated here, the po:update is not
idempotent.

5. Apply this patch
6. Run the gulp update command
7 git commit -a -m"PO update after 34959"
8. Run the gulp update command
9. git diff
=> No changes are generated

Note that this patch will all the entries by files, and per line
numbers.
It fixes a bug in some condition, where we add information/context about
the string. For instance search for "For the first occurrence" in the
file. Prior to this patch this was not correct, we didn't add info about
the first occurrence (but whichever in the list).

Signed-off-by: David Nind <david@davidnind.com>
Comment 6 David Nind 2023-10-10 19:18:11 UTC
I'm not sure whether this bug is related, but have added anyway: Bug 34983 - Retranslating causes changes in locale_data.json

Feel free to remove if it is not.
Comment 7 Jonathan Druart 2023-10-11 07:27:11 UTC
From gettext's changelog:

Version 0.23 - October 2023
 * msgmerge:
   - The msgmerge option '--sorted-output' is now deprecated.
Comment 8 Kyle M Hall (khall) 2023-10-13 13:14:04 UTC
Created attachment 157082 [details] [review]
Bug 34959: Remove unused sub

Nothing special here, this sub is not used and we can remove it.

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Comment 9 Kyle M Hall (khall) 2023-10-13 13:14:16 UTC
Created attachment 157083 [details] [review]
Bug 34959: Sort PO files correctly

The PO files are not sorted when we update them which leads to
unnecessary changes that are commited: hard to see differences and
make git index grow superfluously.

Test plan:
0. Do not apply this patch
1. gulp po:update --lang es-ES
2. git commit -a -m"First PO update"
3. Run again the gulp update command
4. git diff
=> You have a lot of changes generated here, the po:update is not
idempotent.

5. Apply this patch
6. Run the gulp update command
7 git commit -a -m"PO update after 34959"
8. Run the gulp update command
9. git diff
=> No changes are generated

Note that this patch will all the entries by files, and per line
numbers.
It fixes a bug in some condition, where we add information/context about
the string. For instance search for "For the first occurrence" in the
file. Prior to this patch this was not correct, we didn't add info about
the first occurrence (but whichever in the list).

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Comment 10 Tomás Cohen Arazi (tcohen) 2023-10-18 13:23:27 UTC
Pushed to master for 23.11.

Nice work everyone, thanks!
Comment 11 Fridolin Somers 2023-10-25 07:12:59 UTC
Testing on 23.05.x :

3. Run again the gulp update command
4. git diff

With patch I still get a lot off differences there.

This patches are not be very important for old branches right ?
Comment 12 Jonathan Druart 2023-10-25 07:38:23 UTC
(In reply to Fridolin Somers from comment #11)
> Testing on 23.05.x :
> 
> 3. Run again the gulp update command
> 4. git diff
> 
> With patch I still get a lot off differences there.
> 
> This patches are not be very important for old branches right ?

It is needed for the move to Weblate, it must be backported to *all* stable branches for the November releases.
Comment 13 Fridolin Somers 2023-10-25 08:10:51 UTC
OK on it
Comment 14 Fridolin Somers 2023-10-25 08:11:56 UTC
Pushed to 23.05.x for 23.05.05
Comment 15 Jonathan Druart 2023-10-25 08:35:10 UTC
(In reply to Fridolin Somers from comment #14)
> Pushed to 23.05.x for 23.05.05

*November release* is next month!
Comment 16 Fridolin Somers 2023-10-28 20:57:46 UTC
(In reply to Jonathan Druart from comment #15)
> (In reply to Fridolin Somers from comment #14)
> > Pushed to 23.05.x for 23.05.05
> 
> *November release* is next month!

Ah sorry I thought ASAP before November release ^
Reverted
Comment 17 Pedro Amorim 2023-11-14 12:47:11 UTC
(In reply to Fridolin Somers from comment #16)
> (In reply to Jonathan Druart from comment #15)
> > (In reply to Fridolin Somers from comment #14)
> > > Pushed to 23.05.x for 23.05.05
> > 
> > *November release* is next month!
> 
> Ah sorry I thought ASAP before November release ^
> Reverted

Hi Frido, I see this has been pushed, reverted, and pushed again. Can you please update the "Version(S) released in". Thank you!
Comment 18 Jonathan Druart 2023-11-14 13:25:24 UTC
(In reply to Pedro Amorim from comment #17)
> (In reply to Fridolin Somers from comment #16)
> > (In reply to Jonathan Druart from comment #15)
> > > (In reply to Fridolin Somers from comment #14)
> > > > Pushed to 23.05.x for 23.05.05
> > > 
> > > *November release* is next month!
> > 
> > Ah sorry I thought ASAP before November release ^
> > Reverted
> 
> Hi Frido, I see this has been pushed, reverted, and pushed again. Can you
> please update the "Version(S) released in". Thank you!

Done, my bad. Sorry!
Comment 19 Jonathan Druart 2024-01-10 14:58:26 UTC
*** Bug 24797 has been marked as a duplicate of this bug. ***