Bugzilla – Attachment 26576 Details for
Bug 7939
Separate po files for different MARC dialects
Home
|
New
|
Browse
|
Search
|
[?]
|
Reports
|
Help
|
New Account
|
Log In
[x]
|
Forgot Password
Login:
[x]
[patch]
Bug 7939 - Separate po files for different MARC dialects
Bug-7939---Separate-po-files-for-different-MARC-di.patch (text/plain), 16.34 KB, created by
Bernardo Gonzalez Kriegel
on 2014-03-26 14:43:16 UTC
(
hide
)
Description:
Bug 7939 - Separate po files for different MARC dialects
Filename:
MIME Type:
Creator:
Bernardo Gonzalez Kriegel
Created:
2014-03-26 14:43:16 UTC
Size:
16.34 KB
patch
obsolete
>From 81aeb04189f96a1a0e596dca613d8824892870f3 Mon Sep 17 00:00:00 2001 >From: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> >Date: Sun, 23 Mar 2014 19:01:43 -0300 >Subject: [PATCH] Bug 7939 - Separate po files for different MARC dialects > >This patch implements separate PO files for >different MARC dialects. > >It depends on correct filenames, i.e. it will build PO >files using files with/without "unimarc|normarc|marc21" >on their names. > >Prominent changes: >A) tmpl_process3.pl >three new options: exclude or include certain filenames >and a new source dir that can have multiple values >(I opted to add this last feature and not change current >source dir in order not to brake current usage) > >Minor change on use of msgmerge, added use of msgattrib. >Now there are no backup files (*~) and obsoleted translations >are removed (to produce smaller files) >[this can be on another bug, is not strictly related with >the pourpose of this bug] > >B) LangInstaller.pm >Added definitions to create or update xx-YY-{MARCFLAVOR}.po, >minor change to create and install procedure, and modification >of install procedure to handle multiple target dirs. >Updated documentation. > >C) Standarization of filenames >OPAC prog po file is now xx-YY-opac-prog.po >STAFF po file is now xx-YY-staff-prog.po >MARC dialects po files are xx-YY-marc-{MARCFLAVOUR}.po >[again, renaming not related, can be in it's own bug] > >To test: >1) Update po files for your preferred language, ej. nn-NO >cd misc/translator >perl translate update nn-NO >(do it twice if there are warnings) > >2) Do some copying/renaming >cp po/nn-NO-i-staff-t-prog-v-3006000.po po/nn-NO-marc-UNIMARC.po >cp po/nn-NO-i-staff-t-prog-v-3006000.po po/nn-NO-marc-NORMARC.po >cp po/nn-NO-i-staff-t-prog-v-3006000.po po/nn-NO-marc-MARC21.po >mv po/nn-NO-i-staff-t-prog-v-3006000.po po/nn-NO-staff-prog.po >mv po/nn-NO-i-opac-t-prog-v-3006000.po po/nn-NO-opac-prog.po > >(most MARC dialect strings are on staff, so we use that as basis) > >3) Apply the patch > >4) Update again to fix translation files, verbose >perl translate update nn-NO -v > >5) Install language, verbose, verify translations >perl translate install nn-NO -v > >6) Create translation files >rm po/nn-NO* >perl translate create nn-NO > >we must have this list: >po/nn-NO-marc-MARC21.po >po/nn-NO-marc-NORMARC.po >po/nn-NO-marc-UNIMARC.po >po/nn-NO-opac-bootstrap.po >po/nn-NO-opac-ccsr.po >po/nn-NO-opac-prog.po >po/nn-NO-pref.po >po/nn-NO-staff-help.po >po/nn-NO-staff-prog.po > >[staff-help could be more properly staff-prog-help, >another rename] > >Additional tests: >7) Number of msgids >7.a) Before patch and after upgrade, extract and count msgids >for i in $(ls po/nn-NO-*po); \ >do msginit -i $i -o nn-old.po --no-translator --no-wrap --locale=nn_NO; \ >egrep ^msgid nn-old.po >> old; \ >done >sort old | uniq | tee s-old | wc -l > n-old >s-old: have all msgids >n-old: number of msgids > >7.b) After patch and after creation of new files >Repeat procedure, diferent files (s-new, n-new) > >7.c) Compare (diff s-old snew), they are the same >(save for a strange UNIMARC char in my case, but >it's present on corresponding PO file) > >8) Installed dirs/files >8.a) List of EN dirs/files >cd koha-tmpl >find | egrep "/en/" > en > >8.b) List of nn-NO dirs/files. After patch and language install >cd koha-tmpl >find | egrep "/nn-NO/" | sed 's|/nn-NO/|/en/|' > nn > >8.c) Compare (diff en nn), they are the same >--- > misc/translator/LangInstaller.pm | 116 ++++++++++++++++++++++++++------------ > misc/translator/tmpl_process3.pl | 40 +++++++++---- > 2 files changed, 110 insertions(+), 46 deletions(-) > >diff --git a/misc/translator/LangInstaller.pm b/misc/translator/LangInstaller.pm >index 158ca22..ffeade4 100644 >--- a/misc/translator/LangInstaller.pm >+++ b/misc/translator/LangInstaller.pm >@@ -75,8 +75,8 @@ sub new { > > # Get all available language codes > opendir $fh, $self->{path_po}; >- my @langs = map { ($_) =~ /(.*)-i-opac/ } >- grep { $_ =~ /.*-opac-t-prog/ } readdir($fh); >+ my @langs = map { ($_) =~ /(.*)-pref/ } >+ grep { $_ =~ /.*-pref/ } readdir($fh); > closedir $fh; > $self->{langs} = \@langs; > >@@ -86,12 +86,12 @@ sub new { > { > name => 'OPAC prog', > dir => "$opachtdocs/prog", >- suffix => '-i-opac-t-prog-v-3006000.po', >+ suffix => '-opac-prog.po', > }, > { > name => 'Intranet prog UI', > dir => $context->config('intrahtdocs') . '/prog', >- suffix => '-i-staff-t-prog-v-3006000.po', >+ suffix => '-staff-prog.po', > }, > { > name => 'Intranet prog help', >@@ -110,6 +110,20 @@ sub new { > }; > } > >+ # MARC flavours (hardcoded list) >+ for ( "MARC21", "UNIMARC", "NORMARC" ) { >+ my $dirs = $context->config('intrahtdocs') . '/prog'; >+ opendir $fh, $context->config('opachtdocs'); >+ for ( grep { not /^\.|\.\.|lib$/ } readdir($fh) ) { >+ $dirs .= ' ' . "$opachtdocs/$_"; >+ } >+ push @{$self->{interface}}, { >+ name => "$_", >+ dir => $dirs, >+ suffix => "-marc-$_.po", >+ }; >+ } >+ > bless $self, $class; > } > >@@ -329,29 +343,37 @@ sub install_tmpl { > my ($self, $files) = @_; > say "Install templates" if $self->{verbose}; > for my $trans ( @{$self->{interface}} ) { >- print >- " Install templates '$trans->{name}'\n", >- " From: $trans->{dir}/en/\n", >- " To : $trans->{dir}/$self->{lang}\n", >- " With: $self->{path_po}/$self->{lang}$trans->{suffix}\n" >+ my @t_dirs = split(" ", $trans->{dir}); >+ for my $t_dir ( @t_dirs ) { >+ print >+ " Install templates '$trans->{name}'\n", >+ " From: $t_dir/en/\n", >+ " To : $t_dir/$self->{lang}\n", >+ " With: $self->{path_po}/$self->{lang}$trans->{suffix}\n" > if $self->{verbose}; > >- my $trans_dir = ( $trans->{name} =~ /help/ )?"$trans->{dir}":"$trans->{dir}/en/"; >- my $lang_dir = ( $trans->{name} =~ /help/ )?"$trans->{dir}":"$trans->{dir}/$self->{lang}"; >- $lang_dir =~ s|/en/|/$self->{lang}/|; >- mkdir $lang_dir unless -d $lang_dir; >- my $excludes = ( $trans->{name} =~ /UI/ )?"-x 'help'":""; >- >- system >- "$self->{process} install " . >- "-i $trans_dir " . >- "-o $lang_dir ". >- "-s $self->{path_po}/$self->{lang}$trans->{suffix} -r $excludes" . >- ( >- @$files >- ? ' -f ' . join ' -f ', @$files >- : '' >- ) >+ my $trans_dir = ( $trans->{name} =~ /help/ )?"$t_dir":"$t_dir/en/"; >+ my $lang_dir = ( $trans->{name} =~ /help/ )?"$t_dir":"$t_dir/$self->{lang}"; >+ $lang_dir =~ s|/en/|/$self->{lang}/|; >+ mkdir $lang_dir unless -d $lang_dir; >+ my $excludes = ( $trans->{name} !~ /help/ )?"":"-x 'help'"; >+ my $match = ( $trans->{name} =~ /MARC/ )?"-m \"$trans->{name}\"":""; # for MARC translations >+ my $nomatch = ( $trans->{name} !~ /MARC/ )?"-n \"marc21|unimarc|normarc\"":""; # hardcoded MARC variants >+ >+ system >+ "$self->{process} install " . >+ "-i $trans_dir " . >+ "-o $lang_dir ". >+ "-s $self->{path_po}/$self->{lang}$trans->{suffix} -r " . >+ "$excludes " . >+ "$match " . >+ "$nomatch " . >+ ( >+ @$files >+ ? ' -f ' . join ' -f ', @$files >+ : '' >+ ) >+ } > } > } > >@@ -366,16 +388,22 @@ sub update_tmpl { > " From: $trans->{dir}/en/\n", > " To : $self->{path_po}/$self->{lang}$trans->{suffix}\n" > if $self->{verbose}; >- my $lang_dir = "$trans->{dir}/$self->{lang}"; >- mkdir $lang_dir unless -d $lang_dir; > > my $trans_dir = ( $trans->{name} =~ /help/ )?"$trans->{dir}":"$trans->{dir}/en/"; >- my $excludes = ( $trans->{name} =~ /UI/ )?"-x 'help'":""; >+ my $excludes = ( $trans->{name} !~ /help/ )?"":"-x 'help'"; >+ my $match = ( $trans->{name} =~ /MARC/ )?"-m \"$trans->{name}\"":""; # for MARC translations >+ my $dirs = ( $trans->{name} =~ /MARC/ )?"-d ".join(" -d ",split(" ",$trans->{dir})):""; # for MARC translations; >+ my $nomatch = ( $trans->{name} !~ /MARC/ )?"-n \"marc21|unimarc|normarc\"":""; # hardcoded MARC variants >+ > > system > "$self->{process} update " . > "-i $trans_dir " . >- "-s $self->{path_po}/$self->{lang}$trans->{suffix} -r $excludes" . >+ "-s $self->{path_po}/$self->{lang}$trans->{suffix} -r " . >+ "$excludes " . >+ "$dirs " . >+ "$match " . >+ "$nomatch " . > ( > @$files > ? ' -f ' . join ' -f ', @$files >@@ -409,12 +437,19 @@ sub create_tmpl { > if $self->{verbose}; > > my $trans_dir = ( $trans->{name} =~ /help/ )?"$trans->{dir}":"$trans->{dir}/en/"; >- my $excludes = ( $trans->{name} =~ /UI/ )?"-x 'help'":""; >+ my $excludes = ( $trans->{name} !~ /help/ )?"":"-x 'help'"; >+ my $match = ( $trans->{name} =~ /MARC/ )?"-m \"$trans->{name}\"":""; # for MARC translations >+ my $dirs = ( $trans->{name} =~ /MARC/ )?"-d ".join(" -d ",split(" ",$trans->{dir})):""; # for MARC translations; >+ my $nomatch = ( $trans->{name} !~ /MARC/ )?"-n \"marc21|unimarc|normarc\"":""; # hardcoded MARC variants > > system > "$self->{process} create " . > "-i $trans_dir " . >- "-s $self->{path_po}/$self->{lang}$trans->{suffix} -r $excludes" . >+ "-s $self->{path_po}/$self->{lang}$trans->{suffix} -r " . >+ "$excludes " . >+ "$dirs " . >+ "$match " . >+ "$nomatch " . > ( > @$files > ? ' -f ' . join ' -f ', @$files >@@ -504,17 +539,21 @@ appropriate directory. > > =item translate create F<lang> > >-Create 3 .po files in F<po> subdirectory: (1) from opac pages templates, (2) >-intranet templates, and (3) from preferences. >+Create 4 kinds of .po files in F<po> subdirectory: >+(1) one from each theme on opac pages templates, >+(2) intranet templates and help, >+(3) preferences, and >+(4) one for each MARC dialect. >+ > > =over > >-=item F<lang>-opac.po >+=item F<lang>-opac-{theme}.po > > Contains extracted text from english (en) OPAC templates found in >-<KOHA_ROOT>/koha-tmpl/opac-tmpl/prog/en/ directory. >+<KOHA_ROOT>/koha-tmpl/opac-tmpl/{theme}/en/ directory. > >-=item F<lang>-intranet.po >+=item F<lang>-staff-prog.po and F<lang>-staff-help.po > > Contains extracted text from english (en) intranet templates found in > <KOHA_ROOT>/koha-tmpl/intranet-tmpl/prog/en/ directory. >@@ -525,6 +564,11 @@ Contains extracted text from english (en) preferences. They are found in files > located in <KOHA_ROOT>/koha-tmpl/intranet-tmpl/prog/en/admin/preferences > directory. > >+=item F<lang>-marc-{MARC}.po >+ >+Contains extracted text from english (en) files from opac and intranet, >+related with MARC dialects. >+ > =back > > =item pref-trans update F<lang> >diff --git a/misc/translator/tmpl_process3.pl b/misc/translator/tmpl_process3.pl >index e5df69a..b2af19c 100755 >--- a/misc/translator/tmpl_process3.pl >+++ b/misc/translator/tmpl_process3.pl >@@ -3,6 +3,7 @@ > # Parts copyright 2003-2004 Paul Poulain > # Parts copyright 2003-2004 Jerome Vizcaino > # Parts copyright 2004 Ambrose Li >+# Parts copyrigth 2014 Bernardo Gonzalez Kriegel > > =head1 NAME > >@@ -24,6 +25,7 @@ use VerboseWarnings qw( :warn :die ); > > use vars qw( $in_dir @filenames $str_file $out_dir $quiet ); > use vars qw( @excludes $exclude_regex ); >+use vars qw( @dirs $match $nomatch ); > use vars qw( $recursive_p ); > use vars qw( $pedantic_p ); > use vars qw( $href ); >@@ -144,9 +146,11 @@ sub listfiles { > || (defined $exclude_regex && $dirent =~ /^(?:$exclude_regex)$/)) { > ; > } elsif (-f $path) { >- my $basename = basename $path; >+ my $basename = fileparse( $path ); > push @it, $path > if ( not @$filenames or ( grep { $path =~ /$_/ } @$filenames ) ) >+ and ( not defined $match or $basename =~ /($match)/i ) # files to include >+ and ( not defined $nomatch or $basename !~ /($nomatch)/i ) # files not to include > and (!defined $type || $dirent =~ /\.(?:$type)$/) || $action eq 'install'; > } elsif (-d $path && $recursive_p) { > push @it, listfiles($path, $type, $action, $filenames); >@@ -194,7 +198,13 @@ Create or update PO files from templates, or install translated templates. > If given, only these files will be processed. > -s, --str-file=FILE Specify FILE as the translation (po) file > for input (install) or output (create, update) >- -x, --exclude=REGEXP Exclude files matching the given REGEXP >+ -x, --exclude=REGEXP Exclude dirs matching the given REGEXP >+ -d, --dir=DIRECTORY Get or update strings from DIRECTORY. Can have >+ multiple values. Using this ignore -i >+ -m, --match=REGEXP Pipe (|) separated list. Include only files >+ matching the given REGEXP. >+ -n, --nomatch=REGEXP Pipe (|) separated list. Exclude all files >+ matching the given REGEXP. > --help Display this help and exit > -q, --quiet no output to screen (except for errors) > >@@ -223,6 +233,9 @@ GetOptions( > 'recursive|r' => \$recursive_p, > 'str-file|s=s' => \$str_file, > 'exclude|x=s' => \@excludes, >+ 'dir|d=s' => \@dirs, >+ 'match|m=s' => \$match, >+ 'nomatch|n=s' => \$nomatch, > 'quiet|q' => \$quiet, > 'pedantic-warnings|pedantic' => sub { $pedantic_p = 1 }, > 'help' => \&usage, >@@ -261,7 +274,14 @@ for my $fn ( @filenames ) { > die "You cannot specify input files and directories at the same time.\n" > if -d $fn; > } >-@in_files = listfiles($in_dir, $type, $action, \@filenames); >+if ( @dirs ) { # with any -d ignore -i >+ foreach ( @dirs ) { >+ s/\/$//; >+ @in_files = ( @in_files, listfiles($_, $type, $action, \@filenames)); >+ } >+} else { >+ @in_files = listfiles($in_dir, $type, $action, \@filenames); >+} > > # restores the string list from file > $href = Locale::PO->load_file_ashash($str_file); >@@ -348,7 +368,7 @@ if ($action eq 'create') { > close INPUT; > close OUTPUT; > } >- $st = system("msgmerge -U ".($quiet?'-q':'')." -s $str_file $tmpfile2"); >+ $st = system("msgmerge ".($quiet?'-q':'')." -s $str_file $tmpfile2 -o - | msgattrib --no-obsolete -o $str_file"); > } else { > error_normal "Text extraction failed: $xgettext: $!\n", undef; > error_additional "Will not run msgmerge\n", undef; >@@ -377,10 +397,10 @@ if ($action eq 'create') { > if ( @filenames ) { > my ($tmph3, $tmpfile3) = tmpnam(); > $st = system("msgcat $str_file $tmpfile2 > $tmpfile3"); >- $st = system("msgmerge -U ".($quiet?'-q':'')." -s $str_file $tmpfile3") >+ $st = system("msgmerge ".($quiet?'-q':'')." -s $str_file $tmpfile3 -o - | msgattrib --no-obsolete -o $str_file") > unless $st; > } else { >- $st = system("msgmerge -U ".($quiet?'-q':'')." -s $str_file $tmpfile2"); >+ $st = system("msgmerge ".($quiet?'-q':'')." -s $str_file $tmpfile2 -o - | msgattrib --no-obsolete -o $str_file"); > } > } else { > error_normal "Text extraction failed: $xgettext: $!\n", undef; >@@ -532,10 +552,10 @@ actual work. > > =head1 BUGS > >-xgettext.pl must be present in the current directory; the >-msgmerge(1) command must also be present in the search path. >-The script currently does not check carefully whether these >-dependent commands are present. >+xgettext.pl must be present in the current directory; >+msgmerge(1) and msgattrib(1) commands must also be present >+in the search path. The script currently does not check >+carefully whether these dependent commands are present. > > Locale::PO(3) has a lot of bugs. It can neither parse nor > generate GNU PO files properly; a couple of workarounds have >-- >1.7.9.5
You cannot view the attachment while viewing its details because your browser does not support IFRAMEs.
View the attachment on a separate page
.
View Attachment As Diff
View Attachment As Raw
Actions:
View
|
Diff
|
Splinter Review
Attachments on
bug 7939
:
26530
|
26537
|
26576
|
26974
|
28472
|
36241
|
36378
|
38423