Created attachment 157253 [details] [review] Bug 35079: Rebuild POT files only if necessary or asked explicitely By default, gulp tasks po:create and po:update won't rebuild the POT files if they already exist. Both tasks gained a new option --force-extract to rebuild them unconditionally This makes it possible to create/update PO files for multiple languages sequentially without rebuilding the POT file for each language. For instance: gulp po:update --lang fr-FR # do something with the PO files gulp po:update --lang en-GB # ...
Created attachment 157257 [details] [review] Bug 35079: Replace --force-extract by --generate-pot={always,auto,never} This restores the original behaviour of always building the POT file in order to not break existing workflows Option --force-extract is deleted in favor of a new option --generate-pot that can have 3 values: * always: always build the POT file. This is the default value * auto: build the POT file only if it does not exist * never: never build the POT file
time gulp po:update --lang az-AZ --lang prs --lang mon --lang tl-PH --lang te --lang ms-MY --lang ur-Arab --lang rw-RW --lang pbr --lang he-Hebr --lang ku-Arab --lan g de-CH --lang cy-GB --lang my --lang nl-BE --lang is-IS --lang da-DK --lang ia --lang am-Ethi --lang fr-FR --lang sw-KE --lang ne-NE --lang ru-RU --lang iq-CA --lang si --lang fo-FO --lang fa-Arab --lang bn-IN -- lang uk-UA --lang hu-HU --lang tk --lang ar-Arab --lang as-IN --lang nn-NO --lang lo-Laoo --lang ka --lang ml --lang et --lang sq-AL --lang uz --lang iu-CA --lang sk-SK --lang km-KH --lang tr-TR --lang en-NZ --lan g nl-NL --lang tvl --lang ta-LK --lang hi --lang hyw-Armn --lang pt-PT --lang vi-VN --lang ko-Kore-KP --lang lv --lang it-IT --lang ca-ES --lang fr-CA --lang hy-Armn --lang mr --lang pt-BR --lang mi-NZ --lang sl-S I --lang sv-SE --lang dz --lang th-TH --lang pl-PL --lang zh-Hans-CN --lang oc --lang de-DE --lang es-ES --lang cs-CZ --lang gd --lang nb-NO --lang gl --lang ja-Jpan-JP --lang sr-Cyrl --lang id-ID --lang ga --lang ta --lang bg-Cyrl --lang en-GB --lang tet --lang hr-HR --lang el-GR --lang eo --lang ro-RO --lang fi-FI --lang sd-PK --lang eu --lang zh-Hant-TW --lang kn-Knda --lang be-BY real 0m57.700s time ( gulp po:extract && gulp po:update --generate-pot never --lang az-AZ --lang prs --lang mon --lang tl-PH --lang te --lang ms-MY --lang ur-Arab --lang rw-RW --l ang pbr --lang he-Hebr --lang ku-Arab --lang de-CH --lang cy-GB --lang my --lang nl-BE --lang is-IS --lang da-DK --lang ia --lang am-Ethi --lang fr-FR --lang sw-KE --lang ne-NE --lang ru-RU --lang iq-CA --lang si --lang fo-FO --lang fa-Arab --lang bn-IN --lang uk-UA --lang hu-HU --lang tk --lang ar-Arab --lang as-IN --lang nn-NO --lang lo-Laoo --lang ka --lang ml --lang et --lang sq-AL --lang uz --lang iu-CA --lang sk-SK - -lang km-KH --lang tr-TR --lang en-NZ --lang nl-NL --lang tvl --lang ta-LK --lang hi --lang hyw-Armn --lang pt-PT --lang vi-VN --lang ko-Kore-KP --lang lv --lang it-IT --lang ca-ES --lang fr-CA --lang hy-Armn --la ng mr --lang pt-BR --lang mi-NZ --lang sl-SI --lang sv-SE --lang dz --lang th-TH --lang pl-PL --lang zh-Hans-CN --lang oc --lang de-DE --lang es-ES --lang cs-CZ --lang gd --lang nb-NO --lang gl --lang ja-Jpan-JP - -lang sr-Cyrl --lang id-ID --lang ga --lang ta --lang bg-Cyrl --lang en-GB --lang tet --lang hr-HR --lang el-GR --lang eo --lang ro-RO --lang fi-FI --lang sd-PK --lang eu --lang zh-Hant-TW --lang kn-Knda --lang be -BY) real 0m34.250s
(there is a bug when using gulp po:update without --lang)
Created attachment 157304 [details] [review] Bug 35079: Rebuild POT files only if necessary or asked explicitely By default, gulp tasks po:create and po:update won't rebuild the POT files if they already exist. Both tasks gained a new option --force-extract to rebuild them unconditionally This makes it possible to create/update PO files for multiple languages sequentially without rebuilding the POT file for each language. For instance: gulp po:update --lang fr-FR # do something with the PO files gulp po:update --lang en-GB # ... Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Created attachment 157305 [details] [review] Bug 35079: Replace --force-extract by --generate-pot={always,auto,never} This restores the original behaviour of always building the POT file in order to not break existing workflows Option --force-extract is deleted in favor of a new option --generate-pot that can have 3 values: * always: always build the POT file. This is the default value * auto: build the POT file only if it does not exist * never: never build the POT file Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Note that if you see: Error: Cannot find module 'stream/promises' It means that you need a more recent version of node. I was using v14 on my host (which I don't usually use for that), and it needs at least 15. We have v18 in ktd.
If compatibility with older node.js versions is wanted, it's possible to use the non-promise version. But v14 and v16 are EOL now, so they shouldn't be used anyway.
Logging the following here. There is a bug when you run `gulp po:update` without --lang (or with all langs passed). Error: Command failed: msgmerge --backup=off --no-wrap --quiet -F --update misc/translator/po/en-NZ-messages.po misc/translator/Koha-messages.pot misc/translator/Koha-messages.pot:11:35: syntax error misc/translator/Koha-messages.pot:11: keyword "n" unknown misc/translator/Koha-messages.pot:12: end-of-line within string There is an extra quote at the end of line 11 ("\n" instead of \n"). Doing a po:extract then a po:update seems to fix the problem.
(In reply to Jonathan Druart from comment #9) > There is an extra quote at the end of line 11 ("\n" instead of \n"). 11 "Content-Transfer-Encoding: 8bit"\n"
Passing QA, I need it ASAP.
Pushed to master for 23.11. Nice work everyone, thanks!
Depends on Bug 35024 not yet in 23.05.x
(In reply to Jonathan Druart from comment #7) > Note that if you see: > Error: Cannot find module 'stream/promises' > > It means that you need a more recent version of node. > > I was using v14 on my host (which I don't usually use for that), and it > needs at least 15. > > We have v18 in ktd. I have version 18 nodejs/nodistro,now 18.18.2-1nodesource1 amd64 [installed] But I still get this stream/promises error after build-resources tries to run gulp build && gulp build --view opac Error: Cannot find module 'stream/promises' Require stack: - /usr/share/koha/gulpfile.js - /usr/share/koha/node_modules/gulp-cli/lib/shared/require-or-import.js - /usr/share/koha/node_modules/gulp-cli/lib/versioned/^4.0.0/index.js - /usr/share/koha/node_modules/gulp-cli/index.js - /usr/share/koha/node_modules/gulp/bin/gulp.js at Function.Module._resolveFilename (internal/modules/cjs/loader.js:931:15) at Function.Module._load (internal/modules/cjs/loader.js:774:27) at Module.require (internal/modules/cjs/loader.js:1003:19) at require (internal/modules/cjs/helpers.js:107:18) at Object.<anonymous> (/usr/share/koha/gulpfile.js:11:16) at Module._compile (internal/modules/cjs/loader.js:1114:14) at Object.Module._extensions..js (internal/modules/cjs/loader.js:1143:10) at Module.load (internal/modules/cjs/loader.js:979:32) at Function.Module._load (internal/modules/cjs/loader.js:819:12) at Module.require (internal/modules/cjs/loader.js:1003:19) { code: 'MODULE_NOT_FOUND', requireStack: [ '/usr/share/koha/gulpfile.js', '/usr/share/koha/node_modules/gulp-cli/lib/shared/require-or-import.js', '/usr/share/koha/node_modules/gulp-cli/lib/versioned/^4.0.0/index.js', '/usr/share/koha/node_modules/gulp-cli/index.js', '/usr/share/koha/node_modules/gulp/bin/gulp.js' ] }
(In reply to Marcel de Rooy from comment #14) > I have version 18 > nodejs/nodistro,now 18.18.2-1nodesource1 amd64 [installed] Does `node --version` return 18 ? Maybe you have an older version installed in /usr/local are something like that.
(In reply to Julian Maurice from comment #15) > (In reply to Marcel de Rooy from comment #14) > > I have version 18 > > nodejs/nodistro,now 18.18.2-1nodesource1 amd64 [installed] > > Does `node --version` return 18 ? Maybe you have an older version installed > in /usr/local are something like that. Thats correct (somehow): node --version v14.21.3
Yeah, nice: /usr/bin/node --version v18.18.2
Just adding a few lines for anyone who reads this later. Resolved situation by: Using NODE_MAJOR=18 # npm / nodejs mkdir -p /etc/apt/keyrings; \ curl -fsSL https://deb.nodesource.com/gpgkey/nodesource-repo.gpg.key | gpg --dearmor -o /etc/apt/keyrings/nodesource.gpg; \ echo "deb [signed-by=/etc/apt/keyrings/nodesource.gpg] https://deb.nodesource.com/node_$NODE_MAJOR.x nodistro main" | tee /etc/apt/sources.list.d/nodesource.list; \ apt-get update; \ apt-get -yq install nodejs; \ npm install -g yarn; \ Note that you cannot apt-get install nodejs npm. This gives a conflict. npm is part of the above install of nodejs.
(In reply to Marcel de Rooy from comment #18) > Just adding a few lines for anyone who reads this later. Resolved situation > by: > > Using NODE_MAJOR=18 > > # npm / nodejs > mkdir -p /etc/apt/keyrings; \ > curl -fsSL https://deb.nodesource.com/gpgkey/nodesource-repo.gpg.key | > gpg --dearmor -o /etc/apt/keyrings/nodesource.gpg; \ > echo "deb [signed-by=/etc/apt/keyrings/nodesource.gpg] > https://deb.nodesource.com/node_$NODE_MAJOR.x nodistro main" | tee > /etc/apt/sources.list.d/nodesource.list; \ > apt-get update; \ > apt-get -yq install nodejs; \ > npm install -g yarn; \ > > Note that you cannot apt-get install nodejs npm. This gives a conflict. npm > is part of the above install of nodejs. Yes, see also https://gitlab.com/koha-community/koha-testing-docker/-/issues/404
(In reply to Jonathan Druart from comment #9) > Logging the following here. > > There is a bug when you run `gulp po:update` without --lang (or with all > langs passed). > > Error: Command failed: msgmerge --backup=off --no-wrap --quiet -F --update > misc/translator/po/en-NZ-messages.po misc/translator/Koha-messages.pot > > misc/translator/Koha-messages.pot:11:35: syntax error > > misc/translator/Koha-messages.pot:11: keyword "n" unknown > > misc/translator/Koha-messages.pot:12: end-of-line within string > > > There is an extra quote at the end of line 11 ("\n" instead of \n"). > > Doing a po:extract then a po:update seems to fix the problem. See bug 35332.