String extraction for system preferences puts context inside msgid. For instance: # Accounting > Policy msgid "accounting.pref#FinePaymentAutoPopup# automatically display " "a print dialog for a payment receipt when making a payment.." Context should be put into msgctxt, and the filename should be in a comment, like this: #: koha-tmpl/intranet-tmpl/prog/en/modules/admin/preferences/accounting.pref msgctxt "Accounting > Policy > FinePaymentAutoPopup" msgid "automatically display a print dialog for a payment receipt " "when making a payment.."
Created attachment 110777 [details] [review] Bug 26547: Move context from msgid to msgctxt in pref PO files String extraction for system preferences was putting context inside msgid. For instance: # Accounting > Policy msgid "accounting.pref#FinePaymentAutoPopup# automatically display " "a print dialog for a payment receipt when making a payment.." Now context is put into msgctxt, and the reference is set, which is cleaner #: koha-tmpl/intranet-tmpl/prog/en/modules/admin/preferences/accounting.pref msgctxt "Accounting > Policy > FinePaymentAutoPopup" msgid "automatically display a print dialog for a payment receipt " "when making a payment.." The downside is that some messages will have to be re-translated, especially short messages like 'Do', for which msgmerge has a hard time finding the corresponding new msgid. Test plan: 1. Run `gulp po:update` 2. Verify the contents of updated pref PO files 3. Run `cd misc/translator && ./translate install <lang>` 4. Verify that sysprefs are correctly translated 5. Run `prove t/misc/translator/xgettext-pref.t`
This was initially part of bug 25067, but it is better to have this in a separate bug.
Created attachment 111557 [details] [review] Bug 26547: Move context from msgid to msgctxt in pref PO files String extraction for system preferences was putting context inside msgid. For instance: # Accounting > Policy msgid "accounting.pref#FinePaymentAutoPopup# automatically display " "a print dialog for a payment receipt when making a payment.." Now context is put into msgctxt, and the reference is set, which is cleaner #: koha-tmpl/intranet-tmpl/prog/en/modules/admin/preferences/accounting.pref msgctxt "Accounting > Policy > FinePaymentAutoPopup" msgid "automatically display a print dialog for a payment receipt " "when making a payment.." The downside is that some messages will have to be re-translated, especially short messages like 'Do', for which msgmerge has a hard time finding the corresponding new msgid. Test plan: 1. Run `gulp po:update` 2. Verify the contents of updated pref PO files 3. Run `cd misc/translator && ./translate install <lang>` 4. Verify that sysprefs are correctly translated 5. Run `prove t/misc/translator/xgettext-pref.t` Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Works as expected and makes sense from my perspective. Signing off... if Bernardo or someone else very familiar with translations wanted to add an SO here I'd be happy to call it a PQA.
Before pushing it, it would be nice to find a way to "convert" existing pref PO files without losing any translations. All msgids change. msgmerge try its best to not lose existing translations, but it fails on short messages like "Do"
I feel this needs more work. I tested together with bug 25067 and for de-DE ALL translations are marked fuzzy now. This would require translators in Pootle to update every and each of them. We need to see how we can avoid this. Poedit shows: Translated 0 of 2582 (0%) - Remaining 2582 But the weird thing is: The fuzzy strings are still used in the translations so my system preference editor shows up in German. Often times fuzzy strings can break the translation process/templates, so they should be ignored/fallback to English. I am in a bit of trouble here - I think we need both, bug 25067 and this one to move forward. I'll put this in FQA and put my SO stamps on the other with a note. We also need to take care of the unpackaged dependency on the other bug.
Created attachment 111697 [details] [review] Bug 26547: Move context from msgid to msgctxt in pref PO files String extraction for system preferences was putting context inside msgid. For instance: # Accounting > Policy msgid "accounting.pref#FinePaymentAutoPopup# automatically display " "a print dialog for a payment receipt when making a payment.." Now context is put into msgctxt, and the reference is set, which is cleaner #: koha-tmpl/intranet-tmpl/prog/en/modules/admin/preferences/accounting.pref msgctxt "Accounting > Policy > FinePaymentAutoPopup" msgid "automatically display a print dialog for a payment receipt " "when making a payment.." The downside is that some messages will have to be re-translated, especially short messages like 'Do', for which msgmerge has a hard time finding the corresponding new msgid. Test plan: 1. Run `gulp po:update` 2. Verify the contents of updated pref PO files 3. Run `cd misc/translator && ./translate install <lang>` 4. Verify that sysprefs are correctly translated 5. Run `prove t/misc/translator/xgettext-pref.t` Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Created attachment 111698 [details] [review] Bug 26547: Put msgctxt into a comment as well Pootle does not allow to search a message by context, but it allows to search by comment. Having the context (Tab > Section > Syspref) inside a comment make it easier to translate all messages for a specific syspref.
Created attachment 111699 [details] [review] Bug 26547: Fix msgctxt in pref PO file for blocks without a pref name Example in enhanced_content.pref: Enhanced content: Adlibris: - - <strong>NOTE:</strong> Using resources such as external images might leak sensitive data to third parties.
Created attachment 111700 [details] [review] Bug 26547: *DONT PUSH* Script for updating existing pref PO files The script will convert pref PO files to the new format, with proper msgctxt, reference set, and with a clean msgid It will be needed only once before running `gulp po:update` for the first time after applying patches of bug 26547, so this patch should not be pushed. This step is needed to not lose any translation. Test plan: 1. git checkout master 2. cd misc/translator && ./translate update fr-FR 3. cp po/fr-FR-pref.po /tmp/fr-FR-pref-master.po 4. cp /tmp/fr-FR-pref-master.po /tmp/fr-FR-pref-master-updated.po 5. cd ../.. && misc/devel/update-pref-po-file.pl /tmp/fr-FR-pref-master-updated.po 6. msgcat -s /tmp/fr-FR-pref-master-updated.po > /tmp/fr-FR-pref-master-updated-sorted.po 7. cp /tmp/fr-FR-pref-master-updated.po misc/translator/po/fr-FR-pref.po 8. gulp po:update --lang fr-FR 9. msgcomm -u /tmp/fr-FR-pref-master-updated.po misc/translator/po/fr-FR-pref.po This should print nothing, meaning the result of `gulp:update` is the same as the result of update-pref-po-file.pl 10. diff -U3 /tmp/fr-FR-pref-master-updated-sorted.po misc/translator/po/fr-FR-pref.po The diff should be really small (empty messages removed, differences in line wrapping, but that's all) 11. msgattrib --only-fuzzy /tmp/fr-FR-pref-master.po msgattrib --only-fuzzy /tmp/fr-FR-pref-master-updated.po The output of those two commands should be the same (can be empty if there are no fuzzy messages) 12. cd misc/translator && ./translate install fr-FR Check that preferences are correctly translated in the staff interface For the Release Manager and/or Translation Manager: Once this patchset is pushed, all pref PO files should be updated. This can be done immediately, with updated PO files committed directly in master, or just before sending PO files to Pootle, and they will be committed only before release. It is up to RM/TM. Whatever they choose, the process is the same: 1. Apply this patch 2. Run `misc/devel/update-pref-po-file.pl misc/translator/po/*-pref.po` 3. Run `gulp po:update` 4. Discard this patch 5. Commit or send to pootle
Back to needs signoff. If you want to test, be sure to run misc/devel/update-pref-po-file.pl before any attempt to use gulp po:update
As expected, since this has been in the sign off queue for a while, the patch doesn't apply 8-(.. As there have been a lot of changes to the translation process over the last few releases, I was wondering how this fits in. kohadev-koha@kohadevbox:koha(bz26547)$ git bz apply 26547 Bug 26547 - Move context from msgid to msgctxt in pref PO files 111697 - Bug 26547: Move context from msgid to msgctxt in pref PO files 111698 - Bug 26547: Put msgctxt into a comment as well 111699 - Bug 26547: Fix msgctxt in pref PO file for blocks without a pref name 111700 - Bug 26547: *DONT PUSH* Script for updating existing pref PO files Apply? [(y)es, (n)o, (i)nteractive] y Applying: Bug 26547: Move context from msgid to msgctxt in pref PO files Using index info to reconstruct a base tree... M misc/translator/LangInstaller.pm M misc/translator/xgettext-pref A t/misc/translator/xgettext-pref.t Falling back to patching base and 3-way merge... Auto-merging t/db_dependent/misc/translator/xgettext-pref.t Auto-merging misc/translator/xgettext-pref Auto-merging misc/translator/LangInstaller.pm CONFLICT (content): Merge conflict in misc/translator/LangInstaller.pm error: Failed to merge in the changes. Patch failed at 0001 Bug 26547: Move context from msgid to msgctxt in pref PO files
Adding Jonathan because of his work on Weblate/translations. Only a note here: we need to take care that the Do/Don't etc. are not ending up 'collapsed'. In order to form proper sentences for the system preferences they are not always translated the same way.
Created attachment 190539 [details] [review] Bug 26547: Move context from msgid to msgctxt in pref PO files String extraction for system preferences was putting context inside msgid. For instance: # Accounting > Policy msgid "accounting.pref#FinePaymentAutoPopup# automatically display " "a print dialog for a payment receipt when making a payment.." Now context is put into msgctxt, and the reference is set, which is cleaner #: koha-tmpl/intranet-tmpl/prog/en/modules/admin/preferences/accounting.pref msgctxt "Accounting > Policy > FinePaymentAutoPopup" msgid "automatically display a print dialog for a payment receipt " "when making a payment.." The downside is that some messages will have to be re-translated, especially short messages like 'Do', for which msgmerge has a hard time finding the corresponding new msgid. Test plan: 1. Run `gulp po:update` 2. Verify the contents of updated pref PO files 3. Run `cd misc/translator && ./translate install <lang>` 4. Verify that sysprefs are correctly translated 5. Run `prove t/misc/translator/xgettext-pref.t` Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Created attachment 190540 [details] [review] Bug 26547: Put msgctxt into a comment as well Pootle does not allow to search a message by context, but it allows to search by comment. Having the context (Tab > Section > Syspref) inside a comment make it easier to translate all messages for a specific syspref.
Created attachment 190541 [details] [review] Bug 26547: Fix msgctxt in pref PO file for blocks without a pref name Example in enhanced_content.pref: Enhanced content: Adlibris: - - <strong>NOTE:</strong> Using resources such as external images might leak sensitive data to third parties.
Created attachment 190542 [details] [review] Bug 26547: *DONT PUSH* Script for updating existing pref PO files The script will convert pref PO files to the new format, with proper msgctxt, reference set, and with a clean msgid It will be needed only once before running `gulp po:update` for the first time after applying patches of bug 26547, so this patch should not be pushed. This step is needed to not lose any translation. Test plan: 1. git checkout master 2. cd misc/translator && ./translate update fr-FR 3. cp po/fr-FR-pref.po /tmp/fr-FR-pref-master.po 4. cp /tmp/fr-FR-pref-master.po /tmp/fr-FR-pref-master-updated.po 5. cd ../.. && misc/devel/update-pref-po-file.pl /tmp/fr-FR-pref-master-updated.po 6. msgcat -s /tmp/fr-FR-pref-master-updated.po > /tmp/fr-FR-pref-master-updated-sorted.po 7. cp /tmp/fr-FR-pref-master-updated.po misc/translator/po/fr-FR-pref.po 8. gulp po:update --lang fr-FR 9. msgcomm -u /tmp/fr-FR-pref-master-updated.po misc/translator/po/fr-FR-pref.po This should print nothing, meaning the result of `gulp:update` is the same as the result of update-pref-po-file.pl 10. diff -U3 /tmp/fr-FR-pref-master-updated-sorted.po misc/translator/po/fr-FR-pref.po The diff should be really small (empty messages removed, differences in line wrapping, but that's all) 11. msgattrib --only-fuzzy /tmp/fr-FR-pref-master.po msgattrib --only-fuzzy /tmp/fr-FR-pref-master-updated.po The output of those two commands should be the same (can be empty if there are no fuzzy messages) 12. cd misc/translator && ./translate install fr-FR Check that preferences are correctly translated in the staff interface For the Release Manager and/or Translation Manager: Once this patchset is pushed, all pref PO files should be updated. This can be done immediately, with updated PO files committed directly in master, or just before sending PO files to Pootle, and they will be committed only before release. It is up to RM/TM. Whatever they choose, the process is the same: 1. Apply this patch 2. Run `misc/devel/update-pref-po-file.pl misc/translator/po/*-pref.po` 3. Run `gulp po:update` 4. Discard this patch 5. Commit or send to pootle
It still does not apply for me.
Created attachment 190548 [details] [review] Bug 26547: Move context from msgid to msgctxt in pref PO files String extraction for system preferences was putting context inside msgid. For instance: # Accounting > Policy msgid "accounting.pref#FinePaymentAutoPopup# automatically display " "a print dialog for a payment receipt when making a payment.." Now context is put into msgctxt, and the reference is set, which is cleaner #: koha-tmpl/intranet-tmpl/prog/en/modules/admin/preferences/accounting.pref msgctxt "Accounting > Policy > FinePaymentAutoPopup" msgid "automatically display a print dialog for a payment receipt " "when making a payment.." The downside is that some messages will have to be re-translated, especially short messages like 'Do', for which msgmerge has a hard time finding the corresponding new msgid. Test plan: 1. Run `gulp po:update` 2. Verify the contents of updated pref PO files 3. Run `cd misc/translator && ./translate install <lang>` 4. Verify that sysprefs are correctly translated 5. Run `prove t/misc/translator/xgettext-pref.t` Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Created attachment 190549 [details] [review] Bug 26547: Put msgctxt into a comment as well Pootle does not allow to search a message by context, but it allows to search by comment. Having the context (Tab > Section > Syspref) inside a comment make it easier to translate all messages for a specific syspref.
Created attachment 190550 [details] [review] Bug 26547: Fix msgctxt in pref PO file for blocks without a pref name Example in enhanced_content.pref: Enhanced content: Adlibris: - - <strong>NOTE:</strong> Using resources such as external images might leak sensitive data to third parties.
Created attachment 190551 [details] [review] Bug 26547: *DONT PUSH* Script for updating existing pref PO files The script will convert pref PO files to the new format, with proper msgctxt, reference set, and with a clean msgid It will be needed only once before running `gulp po:update` for the first time after applying patches of bug 26547, so this patch should not be pushed. This step is needed to not lose any translation. Test plan: 1. git checkout master 2. cd misc/translator && ./translate update fr-FR 3. cp po/fr-FR-pref.po /tmp/fr-FR-pref-master.po 4. cp /tmp/fr-FR-pref-master.po /tmp/fr-FR-pref-master-updated.po 5. cd ../.. && misc/devel/update-pref-po-file.pl /tmp/fr-FR-pref-master-updated.po 6. msgcat -s /tmp/fr-FR-pref-master-updated.po > /tmp/fr-FR-pref-master-updated-sorted.po 7. cp /tmp/fr-FR-pref-master-updated.po misc/translator/po/fr-FR-pref.po 8. gulp po:update --lang fr-FR 9. msgcomm -u /tmp/fr-FR-pref-master-updated.po misc/translator/po/fr-FR-pref.po This should print nothing, meaning the result of `gulp:update` is the same as the result of update-pref-po-file.pl 10. diff -U3 /tmp/fr-FR-pref-master-updated-sorted.po misc/translator/po/fr-FR-pref.po The diff should be really small (empty messages removed, differences in line wrapping, but that's all) 11. msgattrib --only-fuzzy /tmp/fr-FR-pref-master.po msgattrib --only-fuzzy /tmp/fr-FR-pref-master-updated.po The output of those two commands should be the same (can be empty if there are no fuzzy messages) 12. cd misc/translator && ./translate install fr-FR Check that preferences are correctly translated in the staff interface For the Release Manager and/or Translation Manager: Once this patchset is pushed, all pref PO files should be updated. This can be done immediately, with updated PO files committed directly in master, or just before sending PO files to Pootle, and they will be committed only before release. It is up to RM/TM. Whatever they choose, the process is the same: 1. Apply this patch 2. Run `misc/devel/update-pref-po-file.pl misc/translator/po/*-pref.po` 3. Run `gulp po:update` 4. Discard this patch 5. Commit or send to pootle
(In reply to Jonathan Druart from comment #18) > It still does not apply for me. Patches can be applied with git-bz. It's a perl-git-bz bug (probably related to encoding)
I am stuck at step 6 of the last patch 6. msgcat -s /tmp/fr-FR-pref-master-updated.po > /tmp/fr-FR-pref-master-updated-sorted.po $ msgcat -s /tmp/fr-FR-pref-master-updated.po > /tmp/fr-FR-pref-master-updated-sorted.po /tmp/fr-FR-pref-master-updated.po:21297: inconsistent use of #~ /tmp/fr-FR-pref-master-updated.po:21305: inconsistent use of #~ /tmp/fr-FR-pref-master-updated.po:21391: inconsistent use of #~ /tmp/fr-FR-pref-master-updated.po:21399: inconsistent use of #~ /tmp/fr-FR-pref-master-updated.po:21407: inconsistent use of #~ /tmp/fr-FR-pref-master-updated.po:21706: duplicate message definition... /tmp/fr-FR-pref-master-updated.po:1078: ...this is the location of the first definition /tmp/fr-FR-pref-master-updated.po:21712: duplicate message definition... /tmp/fr-FR-pref-master-updated.po:1372: ...this is the location of the first definition /tmp/fr-FR-pref-master-updated.po:23381: inconsistent use of #~ /tmp/fr-FR-pref-master-updated.po:23389: inconsistent use of #~ /tmp/fr-FR-pref-master-updated.po:23667: inconsistent use of #~ /tmp/fr-FR-pref-master-updated.po:23675: inconsistent use of #~ /tmp/fr-FR-pref-master-updated.po:23683: inconsistent use of #~ /tmp/fr-FR-pref-master-updated.po:23691: inconsistent use of #~ /tmp/fr-FR-pref-master-updated.po:23699: inconsistent use of #~ /tmp/fr-FR-pref-master-updated.po:24409: inconsistent use of #~ /tmp/fr-FR-pref-master-updated.po:24417: inconsistent use of #~ /tmp/fr-FR-pref-master-updated.po:24425: inconsistent use of #~ /tmp/fr-FR-pref-master-updated.po:24439: inconsistent use of #~ /tmp/fr-FR-pref-master-updated.po:24447: inconsistent use of #~ /tmp/fr-FR-pref-master-updated.po:24455: inconsistent use of #~ msgcat: too many errors, aborting
Created attachment 190613 [details] [review] Bug 26547: Fix compil error Patch from commit 4ee3e62
I think we don't need to adjust msgid. I would remove the comment with a sed before the update. The only problem is removing the fuzzy. We should remove only for the new occurrences, not that ones that were fuzzy before.
Additionally there are some QA failures: [FAIL] misc/translator/LangInstaller.pm FAIL tidiness File is not tidy, please run `perl misc/devel/tidy.pl misc/translator/LangInstaller.pm` [FAIL] t/db_dependent/misc/translator/xgettext-pref.t FAIL tidiness File is not tidy, please run `perl misc/devel/tidy.pl t/db_dependent/misc/translator/xgettext-pref.t` I am also not 100% sure about my "fix compile error", I will need to read again why we added that on bug 31640.