Currently, all languages in installer/data/mysql contain a few subfolders with .sql files mostly containing insert statements. There is a lot of redundancy and maintaining this construction is more work. I propose to use en (English) as a global folder, or even a separate folder global (with English just being one of the languages). The global folder should contain inserts; these sql files should be run first. The translation folder should update only the descriptions etc. This is based on a recent discussion on the dev list.
I am attaching two example patches (they are not completely ready for signoff yet, but they already work). The first patch uses en as global folder and updates the de-DE/mandatory sql files as an example. The second patch uses a new global folder next to the en folder. I will ask for some feedback on the dev list about both approaches. The second approach has a maintenance drawback: we need to maintain updates in the en folder, but the advantage is: it is simpler to add a new language. And it is easier to do specific 'English language' stuff without disturbing the global approach. (Stopwords is obsolete?, but could be an example of it.) The work is done in such a way that we can progressively update all files in the language folders without breaking the current installation process.
Created attachment 19242 [details] [review] Bug 10509: Using updates instead of inserts in translated mysql files This patch makes changes to C4/Installer to allow running global (English) sql inserts before updating them from the specific language file. This patch touches only de-DE/mandatory to illustrate the concept. All language installs should proceed as usual. Test plan: Test the English install and test the German install. If you do not feel confident, check another one too :) Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> Verified that the German install still functions.
Created attachment 19243 [details] [review] Bug 10509: Introducing a global folder next to the English folder Adds a global subfolder in installer/data/mysql. The sql files in that folder are used as inserts. The language folder will be moved to update-style (incl. English). The English folder will be a translation basis. Test plan: Repeat your install now with English and German. Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> Verified that English and German still work with global folder.
Hi to all, after thinking about, I prefer the second approch. But in the language-specific files you can't have only update. We need to permitt also delete and insert. For example in italian sql setup we don't want those two classification ('sudocs', 'SuDoc Classification (U.S. GPO)', 0, 'generic') ('anscr', 'ANSCR (Sound Recordings)', 0, 'generic') And we also will add this one ('indire', 'Classificazione INDIRE', 0, 'generic') But I think it is still possible with this patch
Zeno, it will still be possible to do that (with both methods). jcamins mentioned on irc that we have also 'en' in the templates, why create global here?
Other points from irc discussion: Why not use SOURCE in the sql files to refer to english? Or use a file like 'usesupdates' in the lang folder to tell the installer to run english first? -- idea borrowed from Evergreen CREATE FUNCTION i18n(str TEXT) RETURNS TEXT DETERMINISTIC RETURN str; -- then mark up insert statements with the strings we want to translate INSERT INTO `userflags` VALUES(9,'editcatalogue',i18n('Edit Catalog (Modify bibliographic/holdings data)'),0); -- the i18n() bit is just syntactical sugar to make it easy to extract strings for a POT file -- then code (to be written) would create the language-specific inserts from the PO files INSERT INTO `userflags` VALUES(9,'editcatalogue','Katalogdaten ändern (Titel- und Bestandsdaten ändern)',0);
Well, theoreticaly I prefer a neutral 'global' dir with all standard setup and many different dirs (en, en-US, it-IT, etc) with oly translation and little specific changes. But it si only a preference, I can use every solution.
I was working on another sql file and was tempted to introduce i18n() function to enable translation (in the near future I hope), but as I don't know when or where in the install process this function will be defined, do you think that as an interim solution it's possible to write something like drop function if exists i18n; create function i18n( str TEXT) returns text deterministic return str; in the file? BTW, I think that 'en' works as our global, so no need to add a new dir and less things to change.
(In reply to Bernardo Gonzalez Kriegel from comment #8) > I was working on another sql file and was tempted to introduce i18n() > function to enable translation (in the near future I hope), but as I don't > know when or where in the install process this function will be defined, do > you think that as an interim solution it's possible to write something like > drop function if exists i18n; > create function i18n( str TEXT) returns text deterministic return str; > in the file? I would propose to not combine these things. As the outcome of this discussion has not been crystalized(?) yet, it could only complicate things. Your patch (I suspect for authorities) should just go first. > BTW, I think that 'en' works as our global, so no need to add a new dir and > less things to change. I tend to go in that direction too now, although some have expressed interest in the global direction..
Just thinking about this, we could do something else too :) What do you think about running the English file with i18n calls while defining i18n for the German translation in this case with something as: DELIMITER // DROP FUNCTION IF EXISTS i18n// CREATE FUNCTION i18n (d text) RETURNS text DETERMINISTIC BEGIN RETURN CASE d WHEN 'Remaining circulation permissions' 'Übrige Ausleihberechtigungen' WHEN 'Override blocked renewals' 'Gesperrte Verlängerungen überschreiben' WHEN 'Execute overdue items report' 'Überfälligkeiten-Report ausführen' [ etc. etc.] WHEN 'Configure plugins' 'Plugins konfigurieren' ELSE d END; END // This file could also be very easily created from a POD file..
In that case you will need to call German (or any lang) file first so i18n() function it is correctly defined. That's not reversing the order?
> In that case you will need to call German (or any lang) file first so i18n() > function it is correctly defined. Yes and no :) Actually when running the English sql file that contains the i18n calls, I first pass the German i18n definition to mysql. This will make the English install insert German strings. But you could say that I only have one sql file to run, and I 'source' (read: run) the i18n definition first. For English, I would have a fake i18n that returns the same string, so no translation.
Created attachment 19310 [details] [review] Bug 10509: Using i18n function for translation of MySQL web installer files This patch makes the changes needed to progressively use a i18n function to translate strings in the web installer's MySQL files. The English scripts may contain fields like i18n(columnvalue); for English i18n(a) is a, but for the other languages i18n(a) is the translation of a. The corresponding script in the language folder contains the i18n definition. The work is done in such a way that current installations are not affected. Subsequent patches will make the changes in the .sql files themselves. This can be done per folder, per language. Using the extension .i18n in the last subfolder name triggers the new translation mechanism. (See also test plan.) In Installer.pm the all_languages parameter is removed from load_sql_in_order. This code was not used. This change also affects the call in install.pl. The most important change is the use of a new subroutine _run_mysql_file in load_sql. This new routine contains the above logic. Although I also tested changing marc_framework_sql_list and sample_data_sql_list, it proved imo to be easier (and more maintainable) to do this via load_sql. Test plan: Since no .sql files are changed in this patch, please test if you can install English and e.g. German (or another one) in the web installer without any changes. Bonus points for the following additional tests :) They serve to illustrate the i18n concept more clearly. 1) Place in en/mandatory/userflags.sql around a few strings i18n('...'). Run English install again and check the userflags table. No changes expected. 2) Place in de-DE/mandatory/userflags.sql i18n around a few strings too. Also precede the insert statements with this definition: DROP FUNCTION IF EXISTS i18n; CREATE FUNCTION i18n (d text) RETURNS text DETERMINISTIC RETURN concat(d,'xx'); Now run German install again; some of the userflag strings (with i18n) should now end with xx. 3) Before retesting German install, make some changes: Edit en/optional/sample_holidays.sql. Add i18n calls around strings. Create an additional folder optional.i18n in de-DE. Copy sample_holidays.sql and sample_holidays.txt from de-DE/optional. Edit de-DE/optional.i18n/sample_holidays.sql. Remove all text! Add: DROP FUNCTION IF EXISTS i18n; CREATE FUNCTION i18n (d text) RETURNS text DETERMINISTIC RETURN CASE d WHEN 'Sundays' THEN 'Sonntag' ELSE d END; Now run German install again; in the optional part of the install mark sample holidays (in the last section standing separately; do not confuse with the other one). Check the repeatable_holidays table afterwards. The string above should have been translated only, the others are still English. Undo the changes in .sql files; remove optional.i18n folder.
Created attachment 19311 [details] [review] Bug 10509: Followup for load_sql_in_order in unit tests Although I assume that the unit tests in t/db_dependent/lib are actually obsolete and should probably be removed, the two calls of load_sql_in_order are updated to reflect the current parameter list. (The all_languages parameter has been dropped.)
As we discussed this bug, the solution went in another direction than initially foreseen. I have renamed the bug therefore. This patch provides the foundation for converting the language folders one-by-one to a new style. I will open new reports for the language patches. Plan to first send English and German within some days. You are more than welcome to test this patch and sign it off :)
Created attachment 20076 [details] [review] Bug 10509: Using i18n function for translation of MySQL web installer files This patch makes the changes needed to progressively use a i18n function to translate strings in the web installer's MySQL files. The English scripts may contain fields like i18n(columnvalue); for English i18n(a) is a, but for the other languages i18n(a) is the translation of a. The corresponding script in the language folder contains the i18n definition. The work is done in such a way that current installations are not affected. Subsequent patches will make the changes in the .sql files themselves. This can be done per folder, per language. Using the extension .i18n in the last subfolder name triggers the new translation mechanism. (See also test plan.) In Installer.pm the all_languages parameter is removed from load_sql_in_order. This code was not used. This change also affects the call in install.pl. The most important change is the use of a new subroutine _run_mysql_file in load_sql. This new routine contains the above logic. Although I also tested changing marc_framework_sql_list and sample_data_sql_list, it proved imo to be easier (and more maintainable) to do this via load_sql. Test plan: Since no .sql files are changed in this patch, please test if you can install English and e.g. German (or another one) in the web installer without any changes. Bonus points for the following additional tests :) They serve to illustrate the i18n concept more clearly. 1) Place in en/mandatory/userflags.sql around a few strings i18n('...'). Run English install again and check the userflags table. No changes expected. 2) Place in de-DE/mandatory/userflags.sql i18n around a few strings too. Also precede the insert statements with this definition: DROP FUNCTION IF EXISTS i18n; CREATE FUNCTION i18n (d text) RETURNS text DETERMINISTIC RETURN concat(d,'xx'); Now run German install again; some of the userflag strings (with i18n) should now end with xx. 3) Before retesting German install, make some changes: Edit en/optional/sample_holidays.sql. Add i18n calls around strings. Create an additional folder optional.i18n in de-DE. Copy sample_holidays.sql and sample_holidays.txt from de-DE/optional. Edit de-DE/optional.i18n/sample_holidays.sql. Remove all text! Add: DROP FUNCTION IF EXISTS i18n; CREATE FUNCTION i18n (d text) RETURNS text DETERMINISTIC RETURN CASE d WHEN 'Sundays' THEN 'Sonntag' ELSE d END; Now run German install again; in the optional part of the install mark sample holidays (in the last section standing separately; do not confuse with the other one). Check the repeatable_holidays table afterwards. The string above should have been translated only, the others are still English. Undo the changes in .sql files; remove optional.i18n folder. Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Created attachment 20077 [details] [review] Bug 10509: Followup for load_sql_in_order in unit tests Although I assume that the unit tests in t/db_dependent/lib are actually obsolete and should probably be removed, the two calls of load_sql_in_order are updated to reflect the current parameter list. (The all_languages parameter has been dropped.) Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
This no longer applies because the tests (that weren't being used) have been removed. It would be good to add a test for the 2 new subroutines added.
Comment on attachment 20077 [details] [review] Bug 10509: Followup for load_sql_in_order in unit tests The second patch is obsolete with the removal of the lib/KohaTest suite.
The first patch still applies. Since the second patch only referred to obsolete test units, I reset the status to Signed off. The signoff on the first patch is (imo) not impacted by the removal of the second patch.
(In reply to Chris Cormack from comment #18) > This no longer applies because the tests (that weren't being used) have been > removed. It would be good to add a test for the 2 new subroutines added. Chris, if you refer to the two subroutines _run_mysql_file and _tr_sql_fname, I just consider them as internal 'private' routines (with an underscore) for load_sql_in_order. That would not (always/explicitly) require unit tests. Note that the changes would certainly justify changes in the unit tests for load_sql_in_order, but unfortunately we do not even have a unit test for this routine or its module. Should that block this change?
Changing status to Blocked. Subsequent patches are already in Not apply-status. Apparently, this line of development is not interesting enough to continue.
What is it better to do for develop this patch ?
(In reply to Zeno Tajoli from comment #23) > What is it better to do for develop this patch ? The patches were there, but nothing happened. They do no longer apply (quite fast actually with other changes), etc.
Looks like this depends on bug 10520 which is CLOSED WONTFIX.