Bug 29538

Summary: UNIMARC no longer available as marc flavour when installing in Italian
Product: Koha Reporter: John Fawcett <john>
Component: I18N/L10NAssignee: Bugs List <koha-bugs>
Status: RESOLVED DUPLICATE QA Contact: Testopia <testopia>
Severity: major    
Priority: P5 - low CC: bgkriegel, egpetridis, f.demians, jonathan.druart, thibault.keromnes
Version: unspecified   
Hardware: All   
OS: All   
See Also: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27619
Change sponsored?: --- Patch complexity: ---
Documentation contact: Documentation submission:
Text to go in the release notes:
Version(s) released in:
Bug Depends on: 27621    
Bug Blocks:    

Description John Fawcett 2021-11-20 16:37:22 UTC
Installing 21.05 with it-IT, only MARC21 is available.
Installing 20.05 with it-IT, it is possible to install MARC21 or Unimarc.

To reproduce do a standard package install. I used Ubuntu 21.10
and 
echo 'deb http://debian.koha-community.org/koha 21.05 main' | sudo tee /etc/apt/sources.list.d/koha.list

as repository setup

After mysql and koha-common install and normal configurations and library, I ran
sudo koha-translate --install it-IT

When using the web installer when reaching the marc flavour question, only MARC21 is available.

I noticed in git that 

Koha/installer/data/mysql/it-IT/marcflavour/

which is present in 20.11 is no longer present in 21.05 and language directories have been moved to a localization subdirectory

Koha/installer/data/mysql/localization/it-IT/

where there is no marcflavour subdirectory.

Not sure then why even MARC21 is shown as available given there is no marcflavour directory. 

What needs to be done to put UNIMARC installatino support back into the Italian translation?
Comment 1 Jonathan Druart 2021-11-22 11:30:56 UTC
Hi John,

I have contacted you some months ago and you told me you weren't using them.
Yes, we have removed the it-IT installer files (bug 27621) to make them use the normal translation process. The downside is that fr-FR has not been removed yet (bug 27619), and so UNIMARC frameworks are not available for other languages.

Bernardo is working on it but it's long and tedious.
Comment 2 John Fawcett 2021-11-22 18:42:26 UTC
Hi John
probably I misunderstood. I agreed with the suggestion to remove the language specific installer files in favour of using the english .yml file that would be translated on pootle, since this would mean being able to keep the Italian version more aligned to data structure changes in the English version.

I was not aware that this meant that only MARC21 would be supported for Italian. I assumed that UNIMARC would also be supported.

Is it possible to get UNIMARC back? Is it just a question of doing the translation on pootle and getting up published? If so I am more than willing to help out as I imagine other Italian users are too.

Thanks
John
Comment 3 John Fawcett 2021-11-23 22:04:45 UTC
Jonathan (sorry I called you John on the previous update!)

just to clarify a bit more. I don't believe that removal of UNIMARC support from Italian localization was ever raised or discussed (or agreed). Please correct me if I got this wrong.

I took a look on Pootle and I don't see any missing translation for Italian UNIMARC support.

So I do believe that this is a bug. If files about the Italian localization have been deleted without introducing the alternative mechanism, then this is removing a feature. I'm not sure if this is a temporary removal or not, since I don't understand what would be needed to add back UNIMARC support and what help might be needed to do that.

My suggestion is to outline what is the strategy for UNIMARC support in non English versions and if that is not something immediate (you mentioned it was long and tedious) then I suggest bringing back the localization files that were deleted (and updating them to latest En UNIMARC code which I can help with) until such time as the strategy can be implemented.

thanks
John
Comment 4 Jonathan Druart 2021-11-24 09:05:35 UTC
I am sorry John for the misunderstanding. I thought you didn't need the UNIMARC frameworks from the Koha codebase because you were using yours.

At the moment there is no way to have the UNIMARC frameworks installed in it-IT for 21.05 and 21.11 (or any other languages that fr-FR).
But I will do my best to make them back in 22.05

It was a temporary removal anyway, the plan is to:
1. build the UNIMARC frameworks that are as close as possible to IFLA's
2. move them to the general directory to make them available for all languages
3. have the framework strings translatable on pootle

I am in contact with Bernardo and will get back to you as soon as possible. I am focused on the 21.11 release right now but will hopefully have time to dedicate to this after the release.

We will definitely need more hands to translate and check the process, so it's good to know you will be there!
Comment 5 Jonathan Druart 2021-11-24 09:07:00 UTC
(note that you can still install 20.11 it-IT with the UNIMARC frameworks and do a regular update to 21.05 or 21.11)
Comment 6 John Fawcett 2021-11-24 20:44:33 UTC
Jonathan

The workaround sounds useful and hopefully means that effort can more profitably directed to the target solution rather than reverting changes.

I appreciate that there are other priorities and that it may be a big job. But just to understand what's needed: the en UNIMARC directories are still sql based, so looks like first step is to move to .yml configurations. 

Then I think the task is to create files like these:
/usr/share/koha/misc/translator/po/en-GB-installer-UNIMARC.po
/usr/share/koha/misc/translator/po/it-IT-installer-UNIMARC.po

Is that correct or am I underestimating the impacts? Are any updates needed to core code. I was assuming no, since the Italian MARC21 is getting installed with the localization strings, which I assume are coming from the combination of yml and /usr/share/koha/misc/translator/po/it-IT-installer-MARC21.po, since there are no longer specific sql files for it-IT.

I guess, given there is a working version of it-IT UNIMARC on previous versions, that some kind of scripting can be done to create the yml and master pot files and draft translated po files from existing data. Do you already have such tools that maybe have been used to create MARC21 localization or is it all hand written?

Thanks
John
Comment 7 John Fawcett 2021-11-25 00:51:08 UTC
Sorry I meant:

to create UNIMARC localization
Comment 8 Jonathan Druart 2021-11-25 10:43:31 UTC
We need to create a UNIMARC framework that does not contain specificities, only things coming from the standard. Then yes, we need to build a yml file (Bernardo has scripts for that already I think).

Once we have that (for biblios and authorities), we need to add them to installer/data/mysql/en/marcflavour/unimarc/ and it will then be available for the different languages.
The translations will be then done on pootle, and it-IT-installer-UNIMARC.po will contain the strings when a new version is released.

As I told you, Bernardo was working on it a couple of months ago, so some work have been done already. I am waiting for an answer from him to know what is still missing to provide a patch here.
Comment 9 Manos PETRIDIS 2021-12-04 18:11:25 UTC
Am I correct to understand that Jonathan Druart decided to remove functionality from the koha-common package after consulting a single person? Are there no other libraries in Italy following the UNIMARC standard? Are there no other libraries in Italy interested in installing koha?
Comment 10 John Fawcett 2021-12-05 08:17:02 UTC
Manos. 
no, there was no discussion about removing UNIMARC. The discussion was about a more maintainable way of providing UNIMARC translations that keep in sync with the original template. With the then existing mechanism UNIMARC was no longer installing anyway for Italian and I submitted some fixes to realign the translated version with the baseline which meant that it was now installable again. (From this I gather that not many new installations were being done in Italian for UNIMARC, but not sure whether people didn't need it or they needed it but it wasn't working).

The basic problem with the then existing approach is translations were in the code. Jonathan proposed an alternative approach which would mean all translations would be on pootle and there was only one version of the template in the code (so nothing in the code to get out of sync as had happened). I believe that is a good approach and gave my feedback on it.

I don't know who was involved in the decision but I do know I was not the only person to be asked for feedback.

Your question is related to the Italian version, but I believe the modification applies to all translated versions except English and French.

I think the issue is not about the discussion that was done or with how many people it was done, but that not all the code changes needed were committed together. As Jonathan has said, there is however a workaround, since upgrades from previouos versions installed with UNIMARC are supported.

John
Comment 11 Jonathan Druart 2022-03-28 07:43:49 UTC

*** This bug has been marked as a duplicate of bug 30373 ***