Bug 27236

Summary: Package PO translation files separately to koha-common
Product: Koha Reporter: David Cook <dcook>
Component: PackagingAssignee: Bugs List <koha-bugs>
Status: RESOLVED DUPLICATE QA Contact: Testopia <testopia>
Severity: enhancement    
Priority: P5 - low CC: jonathan.druart, mtj
Version: Main   
Hardware: All   
OS: All   
See Also: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24211
Change sponsored?: --- Patch complexity: ---
Documentation contact: Documentation submission:
Text to go in the release notes:
Version(s) released in:
Circulation function:

Description David Cook 2020-12-15 01:33:06 UTC
I propose that we package PO translation files separately as "koha-i18n".

This is a common practice in Debian: https://packages.debian.org/search?searchon=names&keywords=i18n

As Bernardo pointed out in Bug 24211, ~350MB of Koha's installed size is just PO files. That's roughly 75% of Koha's installed size.

Combined with Bernardo's early ideas from Bug 24211, we could actually package the PO files as compressed files (down from ~350MB to ~50MB).

Libraries that need translations could make sure their vendors install koha-i18n (this is another use-case for koha-core vs koha-full), and then they can just decompress the languages required.

Technically, this might even make it easier to provide updated translations when the only change in Koha is the translation (and no code/template changes).
Comment 1 Jonathan Druart 2020-12-15 08:27:57 UTC
That's something I suggested few months ago:
"""Also, maybe later, koha-l10n (or even koha-l10n-es-ar, koha-l10n-es-es, etc.)"""

And that's related to https://wiki.koha-community.org/wiki/Git_Splitting_and_Shrinking (which I am planning to work on this dev cycle).

We can even imagine language packs to download that can be installed by (super)librarians.
Comment 2 Katrin Fischer 2020-12-15 18:47:04 UTC
Please keep in mind that the translations only work well when po files and Koha version match exactly. So if we provided language packs for download, we'd have to provide a LOT of them - for every language and every bugfix release. It might add possible errors over having the files included.

At the moment, when updating Koha, it will automatically also update and rerun your translations. This feature needs to be kept if we change the workflow. There should not be any extra steps required for admins once the language has been installed and activated.

Also: Translations are needed at installation time already with the changes to the installer, so we need to make this really clear that you need to install extra stuff.

I see the reason for shrinking the size, but we need to avoid making translated Koha a second class citizen.
Comment 3 David Cook 2020-12-15 22:20:36 UTC
(In reply to Jonathan Druart from comment #1)
> That's something I suggested few months ago:

Great minds think alike ;)

> """Also, maybe later, koha-l10n (or even koha-l10n-es-ar, koha-l10n-es-es,
> etc.)"""
> 
> And that's related to
> https://wiki.koha-community.org/wiki/Git_Splitting_and_Shrinking (which I am
> planning to work on this dev cycle).
> 
> We can even imagine language packs to download that can be installed by
> (super)librarians.

That sounds pretty cool.
Comment 4 David Cook 2020-12-15 22:28:06 UTC
(In reply to Katrin Fischer from comment #2)
> Please keep in mind that the translations only work well when po files and
> Koha version match exactly. 

I've had mixed results with that in the past. I recall there even being advice in the past to sometimes download newer PO files because those bundled with Koha were wrong. I recall years ago having a really bad experience with the Arabic PO files...

> So if we provided language packs for download,
> we'd have to provide a LOT of them - for every language and every bugfix
> release. It might add possible errors over having the files included.

If we did it via Debian packaging, it would be automated, so it shouldn't be a drama. But if we did provide language packs via a different non-automated mechanism, I could certainly see that being an issue. But we could build something to handle that.

> At the moment, when updating Koha, it will automatically also update and
> rerun your translations. This feature needs to be kept if we change the
> workflow. 

Yep, I had that in mind.

> There should not be any extra steps required for admins once the
> language has been installed and activated.
> 

That sounds reasonable.

> Also: Translations are needed at installation time already with the changes
> to the installer, so we need to make this really clear that you need to
> install extra stuff.
 
Now that could be a problem. What do you mean by "installation" time? Do you mean installing the Debian package, or do you mean "koha-create" or the web installer or something else?
Comment 5 David Cook 2020-12-15 22:30:15 UTC
> I see the reason for shrinking the size, but we need to avoid making
> translated Koha a second class citizen.

Very true. And I do have that in mind too. While the majority of my installations are English-only, I do support translated Koha instances too, and I know a huge percent of Koha instances around the world are translated Koha instances. 

Actually, I'd love to see translated Koha become easier.

That brings me to an alternative idea... where we don't deploy PO files at all, but rather package translated templates and deploy those. (I think this would work well with the language pack idea.)
Comment 6 David Cook 2020-12-15 22:34:24 UTC
> I see the reason for shrinking the size, but we need to avoid making
> translated Koha a second class citizen.

Another way to do that would be to include English "en" in the packaging of languages.

We could make koha-full/koha-common include koha-l10n-en, but then koha-core could have no templates included, so that people could install just the templates that they want.

Actually, if we were packaging templates rather than PO files, it could actually make sense to package "themes". 

But now I'm just going down a rabbit hole of ideas.
Comment 7 David Cook 2020-12-15 22:39:19 UTC
(In reply to Katrin Fischer from comment #2)
> I see the reason for shrinking the size, but we need to avoid making
> translated Koha a second class citizen.

With the "koha-i18n" idea, it would be a dependency of koha-common/koha-full, so in theory it would be the exact same experience as now.

However, people could choose to install koha-core instead, and then not install "koha-i18n" if they're English-only. (This also opens up room for Jonathan's language pack idea.) It would be for advanced installations.

Regarding translated Koha, remember too that most libraries will use a maximum of what... 5 translations (I think that our anecdotal max is 3-4)? The vast majority of PO files will never ever be used by any installed Koha. 

Most software I use on a daily basis allows you to only locally install the languages you use. I tend to use English, French, Chinese, and Korean on my phone for instance (where I had to install French/Chinese/Korean manually). I have no need of the vast majority of other languages.
Comment 8 Katrin Fischer 2020-12-15 23:48:44 UTC
To answer your question about the po files: currently and for a long time we include the right po files or each version reliably in the releases. Downloading from Pootle is most likely to break more things than help you because the versions will probably not match (or will only match for a timeframe of like 4 weeks).
Comment 9 David Cook 2022-12-06 06:38:36 UTC
I think that this one might be too difficult to achieve
Comment 10 Jonathan Druart 2022-12-06 07:12:25 UTC
This is still a goal we should aim for in my opinion.
Comment 11 David Cook 2023-10-29 22:39:06 UTC

*** This bug has been marked as a duplicate of bug 35174 ***