Attachment 120685 Details for Bug 28291 – Bug 28291: Make koha-translate script produce valid UTF-8 YAML files

[patch] Bug 28291: Make koha-translate script produce valid UTF-8 YAML files

Bug-28291-Make-koha-translate-script-produce-valid.patch (text/plain), 2.36 KB, created by Kyle M Hall (khall) on 2021-05-07 11:44:48 UTC

(hide)

Description:

Filename:

MIME Type:

Creator: Kyle M Hall (khall)

Created: 2021-05-07 11:44:48 UTC

Size: 2.36 KB

patch

obsolete

>From b26f1ca3beb800a6e84536c62b5932fd31629b5e Mon Sep 17 00:00:00 2001
>From: =?UTF-8?q?Joonas=20Kylm=C3=A4l=C3=A4?= <joonas.kylmala@helsinki.fi>
>Date: Wed, 5 May 2021 17:20:38 +0300
>Subject: [PATCH] Bug 28291: Make koha-translate script produce valid UTF-8
> YAML files
>
>We had two problems in the LangInstaller.pm module:
> 1) the PO file was read as bytes instead of as a UTF-8 text stream
> 2) The YAML file being outputted was double encoded, once by setting
>    the file handle to output UTF-8 and other time in the DumpFile
>    function internally
>
>To test:
>1. Before applying patch do the following
>$ cd misc/translator
>$ ./translate update pl-PL
>$ ./translate install pl-PL
>$ cd -
>$ less installer/data/mysql/pl-PL/marcflavour/marc21/mandatory/authorities_normal_marc21.yml
>2. Notice the output of the authorities_normal_marc21.yml contains
>invalid looking UTF-8 characters
>3. $ git clean -d -f # to remove the old translation files
>4. Apply patch and repeat the steps and notice the
>authorities_normal_marc21.yml contains valid looking UTF-8 characters
>
>Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
>---
> misc/translator/LangInstaller.pm | 6 ++----
> 1 file changed, 2 insertions(+), 4 deletions(-)
>
>diff --git a/misc/translator/LangInstaller.pm b/misc/translator/LangInstaller.pm
>index d15d15a82c..03e4195f22 100644
>--- a/misc/translator/LangInstaller.pm
>+++ b/misc/translator/LangInstaller.pm
>@@ -306,7 +306,7 @@ sub translate_yaml {
>     my $po_file = $self->po_filename( $target->{suffix} );
>     return $srcyml unless ( -e $po_file );
> 
>-    my $po_ref  = Locale::PO->load_file_ashash( $po_file );
>+    my $po_ref  = Locale::PO->load_file_ashash( $po_file, 'utf8' );
> 
>     my $dstyml   = YAML::XS::LoadFile( $srcyml );
> 
>@@ -400,9 +400,7 @@ sub install_installer {
>             for my $file ( @files ) {
>                 if ( $file =~ /yml$/ ) {
>                     my $translated_yaml = translate_yaml( $self, $target, "$intradir/$dir/$file" );
>-                    open(my $fh, ">:encoding(UTF-8)", "$intradir/$tdir/$file");
>-                    YAML::XS::DumpFile( $fh, $translated_yaml );
>-                    close($fh);
>+                    YAML::XS::DumpFile( "$intradir/$tdir/$file", $translated_yaml );
>                 } else {
>                     File::Copy::copy( "$intradir/$dir/$file", "$intradir/$tdir/$file" );
>                 }
>-- 
>2.24.3 (Apple Git-128)

Actions: View | Diff | Splinter Review

Attachments on bug 28291: 120532 | 120624 | 120685