Many changes for Ukrainian and Russian sql: * update (and translate) many sql-tables * reformatting unimarc-tables, split to local and general fields/subfields * global chande to use 995 subfields for items (952 now in obsolete) * initial support for marc21 (Russian and Ukrainian translation) * other updates, needed for 3.0.x > 3.2.x > 3.4.x
Created attachment 4317 [details] [review] archived patch for upgrade Ukrainian and Russian sql (not archived is too big)
I want to upgrade most sql-files for Ukrainian and Russian localizations. Process of submitting the patch is stopped at the next step: git send-email -to koha-patches@lists.koha-community.org kohaclone/0001-Many-changes-for-Ukrainian-and-Russian-sql.patch fatal: kohaclone/0001-Many-changes-for-Ukrainian-and-Russian-sql.patch: 1730: patch contains a line longer than 998 characters warning: no patches were sent I shortened the length of lines in some files but it still does not help. This sql-tables was adapted for Koha 3.2.x and now for Koha 3.4.1. System prefs synchonized with [koha.git]/installer/data/mysql/en/mandatory/sysprefs.sql few days ago. Please commit this patch. Thanks!
Your patch doesn't apply on HEAD. INSERT INTO authorised_values (category, authorised_value, lib) VALUES error: patch failed: installer/data/mysql/ru-RU/optional/patron_atributes.sql:1 error: installer/data/mysql/ru-RU/optional/patron_atributes.sql: patch does not apply error: patch failed: installer/data/mysql/uk-UA/optional/patron_atributes.sql:1 error: installer/data/mysql/uk-UA/optional/patron_atributes.sql: patch does not apply Did you hand edit your patch? It does not apply to blobs recorded in its index. Cannot fall back to three-way merge.
(In reply to comment #3) > Your patch doesn't apply on HEAD. > > INSERT INTO authorised_values (category, authorised_value, lib) VALUES > error: patch failed: installer/data/mysql/ru-RU/optional/patron_atributes.sql:1 > error: installer/data/mysql/ru-RU/optional/patron_atributes.sql: patch does not > apply > error: patch failed: installer/data/mysql/uk-UA/optional/patron_atributes.sql:1 > error: installer/data/mysql/uk-UA/optional/patron_atributes.sql: patch does not > apply > Did you hand edit your patch? > It does not apply to blobs recorded in its index. > Cannot fall back to three-way merge. I not manually edited the patch. I do not understand why there is an error with these files - they are quite simple (look https://www.dropbox.com/s/f07d6nidu39mvdj/uk-UA/optional/patron_atributes.sql and https://www.dropbox.com/s/f07d6nidu39mvdj/ru-RU/optional/patron_atributes.sql). Could something be wrong when I work with GIT..? My steps: 1) git clone git://git.koha-community.org/koha.git kohaclone 2) cd kohaclone git checkout -b dubyk_master origin/master 3) delete kohaclone/installer/data/mysql/uk-UA/ and kohaclone/installer/data/mysql/ru-RU/ 4) Copy my above updated folders uk-UA and ru-RU to kohaclone/installer/data/mysql/ 5) git commit -a 6) git format-patch -s --subject-prefix='PATCH] [SIGNED-OFF' origin/master 7) In result I have file: kohaclone/0001-Many-changes-for-Ukrainian-and-Russian-sql.patch which I tried to send (git send-email -to koha-patches@lists.koha-community.org ...) and attached here. Folders uk-UA/, ru-RU/ and related archives I posted also in Dropbox - https://www.dropbox.com/s/f07d6nidu39mvdj#view:list
Maybe this patch should be divided into many small? With the help "git send-email" I can't do next changes in sql-tables: 1) Delete old tables uk-UA / mandatory / system_preferences_full_optimal_for_install_only.sql uk-UA / mandatory / system_preferences_only_translation_updating.sql uk-UA / mandatory / system_preferences_optimal_values_insert_only.sql ru-RU / mandatory / system_preferences_full_optimal_for_install_only.sql ru-RU / mandatory / system_preferences_only_translation_updating.sql ru-RU / mandatory / system_preferences_optimal_values_insert_only.sql and replace them by new ones https://www.dropbox.com/s/f07d6nidu39mvdj/uk-UA/mandatory/system_preferences.sql https://www.dropbox.com/s/f07d6nidu39mvdj/ru-RU/mandatory/system_preferences.sql 2) Replace old uk-UA / mandatory / sample_notices.sql ru-RU / mandatory / sample_notices.sql with new https://www.dropbox.com/s/f07d6nidu39mvdj/ru-RU/mandatory/sample_notices.sql https://www.dropbox.com/s/f07d6nidu39mvdj/uk-UA/mandatory/sample_notices.sql 3) maybe some other sql-tables - I do not know how to see all files that are not suitable for "git send-email" Other sql-tables I can send with many small patches - if this makes sense?
I try to split this patch into smaller pieces and include them here ...
> I try to split this patch into smaller pieces and include them here ... If you can't, just sent me the file by email. I will push it on a git repository as a branch.
Created attachment 8721 [details] [review] Bug 6434 - Ukrainian and Russian MARC sql-tables Changes: Marc21 support, 995 for items (Unimarc), authorities types for both MARC's, many small field/subfield changes
Created attachment 8722 [details] Bug 6434 - Ukrainian and Russian MARC sql-tables Changes: Marc21 support, 995 for items (Unimarc), authorities types for both MARC's, many small field/subfield changes
Serhij, could say more about what your patch is supposed to do? Why is there two patches attached and not one? (it seems to be the same). Mark as obsolete one of them if necessary. The first patch is non manageable. If it is a bzip2 compressed patch, attach it as you have done for the second patch. I've been able bunzip2 the second patch which delete deprecated Ukrainian and Russian SQL tables but I can't see any new tables.
Oh, I forgot run "git add -A".
Created attachment 8728 [details] Bug 6434 - Ukrainian and Russian MARC sql-tables Changes: Marc21 support, 995 for items (Unimarc), authorities types for both MARC's, many small field/subfield changes
Created attachment 8734 [details] Signed-off patch It works, doesn't break anything, seems to be in Ukrainian/Russian (I can't judge). It's a huge patch (21M). That's why it's attached as a bz2 archive: so the archive must be downloaded, decompressed, and then applied. General remark: Adding this patch to Koha will increase substantially Koha git repository size. It shows that non-code files (.po files, SQL files) could/should be managed outside Koha core repository. See this discussion: http://wiki.koha-community.org/wiki/Git_Splitting_and_Shrinking Here is my test plan: [1] Apply the patch on master [2] Install ru-RU and uk-UA languages: cd misc/translator ./translate ru-RU uk-UA [3] re-create Koha DB: DROP DATABASE koha; CREATE DATABASE koha; [4] Load Koha pro interface: get web installer [5] Choose ru-RU language [6] Everything is in Russian. Click blindly on button until reaching the page where to choose Unimarc/Marc21. [7] Choose Unimarc. [8] Tick on everything to load all UNIMARC SQL files. [9] All SQL are loaded without any error. [10] Check Koha cataloguing. [11] Go to [3] and choose Marc21 at step [7]. [12] Go to [3] and choose uk-UA at step [5]. [13] Go to [3] and choose uk-UA at step [5] and Marc21 at step [7].
Comment on attachment 8734 [details] Signed-off patch Removing patch marker, since this file is actually a .bz2, and cannot be applied like a patch without decompression.
The compressed, signed-off patch only affects SQL files (and their related txt descriptions) in ru-RU and uk-UA. Given the size, and that I'm unfamiliar with Cyrillic, I haven't read the complete patch, but I did a scan for unintended side effects to the database (a limited one), and didn't find any adverse issues. This is a huge patch, decompressed, and would greatly increase Koha's overall size. Our choice, then, is whether to accept this size increase to the repo, forever, or to implement a splitting/shrinking plan and accept the consequences thereof. I would recommend input from as many community members as possible here, both developers and those using non-English translations of Koha. I'm loathe to lose any history from the repo, as it makes my job as a developer harder, but increasing the overall disk space requirement for Koha doesn't do users any favours. I'm not sure I have good advice on how to proceed. From a purely technical standpoint, this file looks safe, and can Pass QA. As to exactly how it's contents are added to Koha, however, may require some additional discussion. Marking as Passed QA, but with the recommendation that we discuss how to apply this before it's pushed.
Work on these tables in partly sponsored by the libraries: LTS Library(http://koha.lts.lviv.ua:8008/,Unimarc), TNTU Scintific Technical Library(http://koha.tntu.edu.ua/,Unimarc), DCU Library (http://109.254.48.77:8008/,Unimarc), Dmitry Pozharsky University Library (http://lib.s-and-e.ru/,Marc21) and ServanTek Company (http://servantek.org.ua/). I not so frequently updating these MARC-sql-table, so there are so many changes. Old tables deleted in this patch (add ~ 2 Mb to patch). To facilitate further support all tables converted as follows: 1) separated standard MARC fields/subfields and specific for Koha 2) reformated all sql as follow (collected the appropriate fields/subfields and aligned): INSERT INTO marc_tag_structure (frameworkcode, tagfield, mandatory, repeatable, liblibrarian, libopac, authorised_value) VALUE ('', '011', '', 1, 'Міжнародний стандартний номер серіального видання (ISSN)', 'ISSN', ''); INSERT INTO marc_subfield_structure (frameworkcode, authtypecode, tagfield, tagsubfield, mandatory, repeatable, liblibrarian, ('', '', '011', 'a', 0, 0, 'Номер (ISSN)', 'ISSN', 0, NULL, 'biblioitems.issn', '', '', NULL, NULL, NULL, ('', '', '011', 'b', 0, 0, 'Уточнення', '', -1, NULL, '', '', '', NULL, NULL, NULL, NULL), ('', '', '011', 'd', 0, 0, 'Умови придбання і/або ціна', '', -1, NULL, '', '', '', NULL, NULL, NULL, NULL), ('', '', '011', 'y', 0, 0, 'Анульований ISSN', '', -1, NULL, '', '', '', NULL, NULL, NULL, NULL), ('', '', '011', 'z', 0, 0, 'Помилковий ISSN', '', -1, NULL, '', '', '', NULL, NULL, NULL, NULL); - for easier reading/editing by human. In result, the size of the tables corresponding to the size of other languages from /data/mysql/ (including the fact that that the Cyrillic characters are 2 bytes). P.S. Also, I have scripts that perform such reformatting as above and the initial script, which compares the fields/subfields for all languages from Koha-sql. If someone is interested, I unload them somewhere publicly.
this patch has been pushed to master on march 31th For an unknown reason, it's not on a new/bug_6434 branch (I suspect it's because it's a zipped patch so I could not use my automated tools with that use git bz and I made a mistake)
This bug will be included in the Koha 3.6.5 release.