Bug 18039

Summary: Move the mandatory and optional files for non-english languages into 'default' directory so bug 17855 permits use of other languages
Product: Koha Reporter: Alex Buckley <alexbuckley>
Component: Installation and upgrade (web-based installer)Assignee: Jonathan Druart <jonathan.druart>
Status: CLOSED INVALID QA Contact: Testopia <testopia>
Severity: enhancement    
Priority: P5 - low CC: aleisha, gmcharlt, jonathan.druart, katrin.fischer, ztajoli
Version: unspecified   
Hardware: All   
OS: All   
Change sponsored?: --- Patch complexity: ---
Documentation contact: Documentation submission:
Text to go in the release notes:
Move the mandatory and optional files for non-english languages into 'default' directory so bug 17855 permits use of other languages
Version(s) released in:
Bug Depends on: 17855, 18456    
Bug Blocks:    
Attachments: Bug 18039: en
Why do we add this empty file?
de-DE
es-ES
fr-CA
fr-FR
it-IT
nb-NO
pl-PL
ru-RU
uk-UA
[DO NOT PUSH] Add move_installer_files.pl
[SIGN-OFF] it-IT
[SIGN-OFF] follow up for it-IT setup
[SIGNED-OFF] it-IT

Description Alex Buckley 2017-02-02 20:02:28 UTC
At the moment after applying bug 17855 (updated web installer and onboarding tool) users can only use Koha in English. 

This bug requires the moving of mandatory and optional files into default directories for all other non-English languages.

The rationale for placing the work for this in a bug separate to bug 17855 is that it is going to take a lot of testing to confirm all files have been moved and Koha with the updated web installer and onboarding tool works with other languages.
Comment 1 Katrin Fischer 2017-02-09 13:45:03 UTC
I think this might be quite a blocker - do the ohter installers still work just without the onboarding? Or does it break them?
What is needed to make them work?
Comment 2 Jonathan Druart 2017-02-09 13:55:37 UTC
It breaks them.
Comment 3 Jonathan Druart 2017-02-09 16:27:49 UTC
Created attachment 60080 [details] [review]
Bug 18039: en

This patches comes from bug 17855.
Comment 4 Jonathan Druart 2017-02-09 16:27:52 UTC
Created attachment 60081 [details] [review]
Why do we add this empty file?

https://bugs.koha-community.org/show_bug.cgi?id=17855
Comment 6 Jonathan Druart 2017-02-09 16:28:19 UTC
Created attachment 60083 [details] [review]
es-ES

Missing:
installer/data/mysql/en/default/csv_profiles.sql

https://bugs.koha-community.org/show_bug.cgi?id=17855
Comment 8 Jonathan Druart 2017-02-09 16:28:36 UTC
Created attachment 60085 [details] [review]
fr-FR

Does not exist for this language
csv_profiles.sql
parameters.sql (currency)
auth_values.sql (YES_NO)

fatal: bad source, source=installer/data/mysql/fr-FR/2-Optionel/auth_val.sql, destination=installer/data/mysql/fr-FR/default/auth_val.sql
fatal: bad source, source=installer/data/mysql/fr-FR/2-Optionel/auth_val.txt, destination=installer/data/mysql/fr-FR/default/auth_val.txt
fatal: bad source, source=installer/data/mysql/fr-FR/2-Optionel/csv_profiles.sql, destination=installer/data/mysql/fr-FR/default/csv_profiles.sql
fatal: bad source, source=installer/data/mysql/fr-FR/2-Optionel/csv_profiles.txt, destination=installer/data/mysql/fr-FR/default/csv_profiles.txt
fatal: bad source, source=installer/data/mysql/fr-FR/2-Optionel/parameters.sql, destination=installer/data/mysql/fr-FR/default/parameters.sql
fatal: bad source, source=installer/data/mysql/fr-FR/2-Optionel/parameters.txt, destination=installer/data/mysql/fr-FR/default/parameters.txt
fatal: bad source, source=installer/data/mysql/fr-FR/1-Obligatoire/auth_values.sql, destination=installer/data/mysql/fr-FR/default/auth_values.sql
fatal: bad source, source=installer/data/mysql/fr-FR/1-Obligatoire/auth_values.txt, destination=installer/data/mysql/fr-FR/default/auth_values.txt
fatal: can not move directory into itself, source=installer/data/mysql/fr-FR/marcflavour/marc21/default/authorities_normal_marc21.sql, destination=installer/data/mysql/fr-FR/marcflavour/marc21/default/authorities_normal_marc21.sql

Replace with
git mv installer/data/mysql/fr-FR/1-Obligatoire/authorised_values.* installer/data/mysql/fr-FR/default

https://bugs.koha-community.org/show_bug.cgi?id=17855
Comment 9 Jonathan Druart 2017-02-09 16:28:43 UTC
Created attachment 60086 [details] [review]
it-IT

fatal: bad source, source=installer/data/mysql/it-IT/necessari/auth_values.sql, destination=installer/data/mysql/it-IT/default/auth_values.sql
fatal: bad source, source=installer/data/mysql/it-IT/necessari/auth_values.txt, destination=installer/data/mysql/it-IT/default/auth_values.txt
fatal: bad source, source=installer/data/mysql/it-IT/necessari/sample_notices.sql, destination=installer/data/mysql/it-IT/default/sample_notices.sql
fatal: bad source, source=installer/data/mysql/it-IT/necessari/sample_notices.txt, destination=installer/data/mysql/it-IT/default/sample_notices.txt

=> OK

https://bugs.koha-community.org/show_bug.cgi?id=17855
Comment 10 Jonathan Druart 2017-02-09 16:28:47 UTC
Created attachment 60087 [details] [review]
nb-NO

csv_profiles does not exist

fatal: bad source, source=installer/data/mysql/nb-NO/2-Valgfritt/csv_profiles.sql, destination=installer/data/mysql/nb-NO/default/csv_profiles.sql
fatal: bad source, source=installer/data/mysql/nb-NO/2-Valgfritt/csv_profiles.txt, destination=installer/data/mysql/nb-NO/default/csv_profiles.txt
fatal: bad source, source=installer/data/mysql/nb-NO/1-Obligatorisk/auth_values.txt, destination=installer/data/mysql/nb-NO/default/auth_values.txt

https://bugs.koha-community.org/show_bug.cgi?id=17855
Comment 11 Jonathan Druart 2017-02-09 16:28:52 UTC
Created attachment 60088 [details] [review]
pl-PL

fatal: bad source, source=installer/data/mysql/pl-PL/optional/csv_profiles.sql, destination=installer/data/mysql/pl-PL/default/csv_profiles.sql
fatal: bad source, source=installer/data/mysql/pl-PL/optional/csv_profiles.txt, destination=installer/data/mysql/pl-PL/default/csv_profiles.txt
fatal: bad source, source=installer/data/mysql/pl-PL/mandatory/auth_values.txt, destination=installer/data/mysql/pl-PL/default/auth_values.txt

https://bugs.koha-community.org/show_bug.cgi?id=17855
Comment 12 Jonathan Druart 2017-02-09 16:29:08 UTC
Created attachment 60089 [details] [review]
ru-RU

fatal: bad source, source=installer/data/mysql/ru-RU/optional/auth_val.sql, destination=installer/data/mysql/ru-RU/default/auth_val.sql
fatal: bad source, source=installer/data/mysql/ru-RU/optional/auth_val.txt, destination=installer/data/mysql/ru-RU/default/auth_val.txt
fatal: bad source, source=installer/data/mysql/ru-RU/optional/csv_profiles.sql, destination=installer/data/mysql/ru-RU/default/csv_profiles.sql
fatal: bad source, source=installer/data/mysql/ru-RU/optional/csv_profiles.txt, destination=installer/data/mysql/ru-RU/default/csv_profiles.txt
fatal: bad source, source=installer/data/mysql/ru-RU/optional/parameters.sql, destination=installer/data/mysql/ru-RU/default/parameters.sql
fatal: bad source, source=installer/data/mysql/ru-RU/optional/parameters.txt, destination=installer/data/mysql/ru-RU/default/parameters.txt
fatal: bad source, source=installer/data/mysql/ru-RU/mandatory/auth_values.txt, destination=installer/data/mysql/ru-RU/default/auth_values.txt

https://bugs.koha-community.org/show_bug.cgi?id=17855
Comment 13 Jonathan Druart 2017-02-09 16:29:15 UTC
Created attachment 60090 [details] [review]
uk-UA

fatal: bad source, source=installer/data/mysql/uk-UA/optional/auth_val.sql, destination=installer/data/mysql/uk-UA/default/auth_val.sql
fatal: bad source, source=installer/data/mysql/uk-UA/optional/auth_val.txt, destination=installer/data/mysql/uk-UA/default/auth_val.txt
fatal: bad source, source=installer/data/mysql/uk-UA/optional/csv_profiles.sql, destination=installer/data/mysql/uk-UA/default/csv_profiles.sql
fatal: bad source, source=installer/data/mysql/uk-UA/optional/csv_profiles.txt, destination=installer/data/mysql/uk-UA/default/csv_profiles.txt
fatal: bad source, source=installer/data/mysql/uk-UA/optional/parameters.sql, destination=installer/data/mysql/uk-UA/default/parameters.sql
fatal: bad source, source=installer/data/mysql/uk-UA/optional/parameters.txt, destination=installer/data/mysql/uk-UA/default/parameters.txt
fatal: bad source, source=installer/data/mysql/uk-UA/mandatory/auth_values.txt, destination=installer/data/mysql/uk-UA/default/auth_values.txt
fatal: bad source, source=installer/data/mysql/uk-UA/mandatory/sample_notices.txt, destination=installer/data/mysql/uk-UA/default/sample_notices.txt

https://bugs.koha-community.org/show_bug.cgi?id=17855
Comment 14 Jonathan Druart 2017-02-09 16:29:22 UTC
Created attachment 60091 [details] [review]
[DO NOT PUSH] Add move_installer_files.pl

https://bugs.koha-community.org/show_bug.cgi?id=17855
Comment 15 Jonathan Druart 2017-02-10 08:00:04 UTC
To generate these patches I have used the move_installer_files.pl script (from the last patch).
We now need to make sure all files have been correctly move for the different languages.
Everything looks ok so far to me, unless ru-RU and uk-UA, that are not really maintained.
Comment 16 Zeno Tajoli 2017-02-25 09:02:13 UTC
Created attachment 60666 [details] [review]
[SIGN-OFF] it-IT
Comment 17 Zeno Tajoli 2017-02-25 09:03:20 UTC
Created attachment 60667 [details] [review]
[SIGN-OFF] follow up for it-IT setup
Comment 18 Alex Buckley 2017-02-26 07:23:17 UTC
All patches apply successfully except for the final patch "[SIGN-OFF] follow up for it-IT setup" which fails to apply with the following error:

>/home/vagrant/kohaclone/.git/rebase-apply/patch:218: trailing whitespace.
>-- 
>/home/vagrant/kohaclone/.git/rebase-apply/patch:222: trailing whitespace>.>
>-- 
>warning: 2 lines add whitespace errors.
>Using index info to reconstruct a base tree...
>M	installer/data/mysql/it-IT/necessari/notices.sql
><stdin>:218: trailing whitespace.
>-- 
><stdin>:222: trailing whitespace.
>-- 
><stdin>:268: new blank line at EOF.
>+
>warning: 2 lines applied after fixing whitespace errors.
>Falling back to patching base and 3-way merge...
>Auto-merging installer/data/mysql/it-IT/default/system_preferences.sql

The file 'notices.sql' file is not being created due to the error.
Comment 19 Zeno Tajoli 2017-02-26 09:01:18 UTC
Hi Alex,

can you explain why you find this error on appling my follow-up to it_IT patch ?

I try to apply the two patches on a clean master/origin git and I don't find any problem.
Comment 20 Alex Buckley 2017-02-26 10:56:34 UTC
(In reply to Zeno Tajoli in comment 19)

Hi Zeno

I have been testing the '[SIGN-OFF] follow up for it-IT setup' patch on a clean branch off master, with the following combinations to see if the error always appeared: 

1. I have tried testing by applying bug 17855 first then all patches of this bug 18039-which threw the aforementioned error

2. I have tried testing by applying 17855 then just the last two patches of this bug- which also threw this error

3. I have also tried applying the last two patches of this bug without 17855 - again the error

I now realise I was incorrect in saying the issue was notices.sql not being created as I now realise it is being moved in this patch to a new directory and so I was looking in the wrong directory. 

However the trailing whitespaces (mentioned in the error) appeared to be in either in notices.sql or system-preferences.sql in lines 218 and 222 and blankline warning in line 268 but neither of these files have the said lines and so I am not sure what file the trailing whitespaces and blank line are in. 

I will ask one of my workmates if they can have a look to see if they can replicate the issues I see.
Comment 21 Jonathan Druart 2017-02-27 08:11:39 UTC
Everything applies cleanly for me.
Comment 22 Jonathan Druart 2017-03-21 13:09:52 UTC
Created attachment 61380 [details] [review]
[SIGNED-OFF] it-IT

fatal: bad source, source=installer/data/mysql/it-IT/necessari/auth_values.sql, destination=installer/data/mysql/it-IT/default/auth_values.sql
fatal: bad source, source=installer/data/mysql/it-IT/necessari/auth_values.txt, destination=installer/data/mysql/it-IT/default/auth_values.txt
fatal: bad source, source=installer/data/mysql/it-IT/necessari/sample_notices.sql, destination=installer/data/mysql/it-IT/default/sample_notices.sql
fatal: bad source, source=installer/data/mysql/it-IT/necessari/sample_notices.txt, destination=installer/data/mysql/it-IT/default/sample_notices.txt

=> OK

https://bugs.koha-community.org/show_bug.cgi?id=17855

https://bugs.koha-community.org/show_bug.cgi?id=18039

it-IT: follow up on [BUG 18039]

For Italy the files notices.sql and system_preferences.sql need to stay under 'default'

Signed-off-by: Zeno Tajoli <z.tajoli@cineca.it>
Comment 23 Jonathan Druart 2017-03-21 13:10:41 UTC Comment hidden (obsolete)
Comment 24 Jonathan Druart 2017-03-21 13:11:05 UTC
Zeno, I have squashed the 2 it-IT patches, with me they size less than 20KB.
Comment 25 Zeno Tajoli 2017-03-21 13:15:32 UTC Comment hidden (obsolete)
Comment 26 Katrin Fischer 2017-08-18 13:26:19 UTC
Patches still apply. 
Is this still needed? 
Won't it require changes in other parts now that the onboarding tool has already been pushed?
Comment 27 Jonathan Druart 2019-02-15 00:41:19 UTC
To clarify, since bug 17855 comment 169 we did not need these files to be moved to get the onboarding tool working for all languages.

The following commit removed the advanced mode.

  commit 3d80ca1758c135042b0ca4d261d1198a40a35197
  Bug 17855 - Reverted back to original step 3 in the web installer so there is no basic and advanced setup mode and consequently no need for SQL files to be moved to a 'default' directory for all languages.



So switched to INVALID...