Bugzilla – Attachment 126328 Details for
Bug 28445
Use the task queue for the batch delete and update items tool
Home
|
New
|
Browse
|
Search
|
[?]
|
Reports
|
Help
|
New Account
|
Log In
[x]
|
Forgot Password
Login:
[x]
[patch]
Bug 28445: Adjust code to handle regexs
Bug-28445-Adjust-code-to-handle-regexs.patch (text/plain), 20.65 KB, created by
Jonathan Druart
on 2021-10-15 14:02:14 UTC
(
hide
)
Description:
Bug 28445: Adjust code to handle regexs
Filename:
MIME Type:
Creator:
Jonathan Druart
Created:
2021-10-15 14:02:14 UTC
Size:
20.65 KB
patch
obsolete
>From 1282f6ad4683e40b2a35bfcd95629684d220154d Mon Sep 17 00:00:00 2001 >From: Jonathan Druart <jonathan.druart@bugs.koha-community.org> >Date: Mon, 26 Jul 2021 18:30:33 +0200 >Subject: [PATCH] Bug 28445: Adjust code to handle regexs > >--- > .../prog/en/includes/html_helpers.inc | 12 +- > .../prog/en/modules/tools/batchMod-edit.tt | 20 +- > tools/batchMod.pl | 277 ++++++++++-------- > 3 files changed, 175 insertions(+), 134 deletions(-) > >diff --git a/koha-tmpl/intranet-tmpl/prog/en/includes/html_helpers.inc b/koha-tmpl/intranet-tmpl/prog/en/includes/html_helpers.inc >index 6aac6ae28ec..0d90b493251 100644 >--- a/koha-tmpl/intranet-tmpl/prog/en/includes/html_helpers.inc >+++ b/koha-tmpl/intranet-tmpl/prog/en/includes/html_helpers.inc >@@ -116,7 +116,9 @@ > [% ELSE %] > <a href="#" id="buttonDot_[%- mv.id | html -%]" class="[%- mv.class | html -%]" title="Tag editor">...</a> > [% END %] >- [%- mv.javascript | $raw -%] >+ [% UNLESS no_plugin %][%# FIXME - from batchMod-edit, jQuery is included at the end of the template and cataloguing plugins are not working in this situation %] >+ [%- mv.javascript | $raw -%] >+ [% END %] > [% END %] > [% ELSIF ( mv.type == 'text' ) %] > [% IF mv.readonly %] >@@ -142,9 +144,9 @@ > > [% IF add_regex %] > <span name="regex_fields" style="display: none;"> >- s/<input type="text" id="[% mv.id | html %]" name="regex_search" placeholder="regex pattern" />/ >- <input type="text" id="[% mv.id | html %]" name="regex_replace" placeholder="regex replacement" />/ >- <input type="text" id="[% mv.id | html %]" name="regex_modifiers" placeholder="ig" size="3" /> >+ s/<input type="text" name="[% kohafield | html %]_regex_search" class="regex_search" placeholder="regex pattern" />/ >+ <input type="text" name="[% kohafield | html %]_regex_replace" class="regex_replace" placeholder="regex replacement" />/ >+ <input type="text" name="[% kohafield | html %]_regex_modifiers" class="regex_modifiers" placeholder="ig" size="3" /> > </span> > [% END %] > >@@ -162,7 +164,7 @@ > > [% IF add_regex %] > [% IF (mv.type == 'text' || mv.type == 'text2' || mv.type == 'textarea' ) %] >- <a href="#" name="field_regex" id="[% subfield.id | html %]" >RegEx</a> >+ <a href="#" name="[% kohafield | html %]_field_regex" class="field_regex" id="[% subfield.id | html %]" >RegEx</a> > [% END %] > [% END %] > >diff --git a/koha-tmpl/intranet-tmpl/prog/en/modules/tools/batchMod-edit.tt b/koha-tmpl/intranet-tmpl/prog/en/modules/tools/batchMod-edit.tt >index af9c38414dc..05399cf7e05 100644 >--- a/koha-tmpl/intranet-tmpl/prog/en/modules/tools/batchMod-edit.tt >+++ b/koha-tmpl/intranet-tmpl/prog/en/modules/tools/batchMod-edit.tt >@@ -248,7 +248,8 @@ > <h2>Edit items</h2> > <div class="hint">Checking the box right next to the subfield label will disable the entry and delete the subfield on all selected items. Leave fields blank to make no change.</div> > <fieldset class="rows"> >- [% PROCESS subfields_for_item subfields => subfields, add_regex => 1, add_delete_checkbox => 1 %] >+ [%# no_plugin from batchMod-edit, jQuery is included at the end of the template and cataloguing plugins are not working in this situation %] >+ [% PROCESS subfields_for_item subfields => subfields, add_regex => 1, add_delete_checkbox => 1, no_plugin => 1 %] > </fieldset> > > <fieldset class="rows"> >@@ -310,8 +311,8 @@ > $("input[name='disable_input']").click(function() { > var row = $(this).attr("id"); > row = row.replace("row","hint"); >- var todisable = $(this).parent().find("[name='field_value'],[name='tag'],[name='subfield'],[name='mandatory']"); >- var regex_link = $(this).parent().find("[name='field_regex']"); >+ var todisable = $(this).parent().find(".input_marceditor,.tag,.subfield,.mandatory"); >+ var regex_link = $(this).parent().find(".field_regex"); > if ($(this).is(":checked")) { > $(todisable).prop('disabled', true); > $("#"+row).html(_("This subfield will be deleted")); >@@ -322,12 +323,11 @@ > $(regex_link).show(); > } > }); >- $('a[name="field_regex"]').click(function() { >- var id = $(this).attr('id'); >- var editor = $(this).parent().find("[name='field_value']"); >+ $('a.field_regex').click(function() { >+ var editor = $(this).parent().find(".input_marceditor"); > var tag_editor = $(this).parent().find(".buttonDot"); > var regex = $(this).parent().find("[name='regex_fields']"); >- var disable_input = $(this).parent().find("[name='disable_input']"); >+ var disable_input = $(this).parent().find(".disable_input"); > if ($(this).html() == 'RegEx') { > $(editor).hide(); > $(regex).show(); >@@ -348,10 +348,10 @@ > [% IF ( show ) %] > [%- # Loop over fields which may have plugin JavaScript -%] > [%- UNLESS (too_many_items_process) -%] >- [%- FOREACH ite IN item -%] >- [% SET mv = ite.marc_value %] >+ [%- FOREACH subfield IN subfields -%] >+ [% SET mv = subfield.marc_value %] > [%- IF ( mv.type == 'text2' ) -%] >- <!-- subfield[% ite.tag | html %][% ite.subfield | html %][% ite.random | html %] --> >+ <!-- subfield[% subfield.tag | html %][% subfield.subfield | html %][% subfield.random | html %] --> > [% mv.javascript | $raw %] > [%- END -%] > [%- END -%] >diff --git a/tools/batchMod.pl b/tools/batchMod.pl >index 13ecde73476..acbbf295a64 100755 >--- a/tools/batchMod.pl >+++ b/tools/batchMod.pl >@@ -117,48 +117,20 @@ if ($op eq "action") { > my @searches = $input->multi_param('regex_search'); > my @replaces = $input->multi_param('regex_replace'); > my @modifiers = $input->multi_param('regex_modifiers'); >- my @disabled = $input->multi_param('disable_input'); >- >- # Is there something to modify ? >- # TODO : We shall use this var to warn the user in case no modification was done to the items >- my $values_to_modify = scalar(grep {!/^$/} @values) || scalar(grep {!/^$/} @searches); >- my $values_to_blank = scalar(@disabled); >- >- my $marcitem; >- >- #initializing values for updates >- my ( $itemtagfield, $itemtagsubfield) = &GetMarcFromKohaField( "items.itemnumber" ); >- if ($values_to_modify){ >- my $xml = TransformHtmlToXml(\@tags,\@subfields,\@values,undef,undef, 'ITEM'); >- $marcitem = MARC::Record::new_from_xml($xml, 'UTF-8'); >- } >- if ($values_to_blank){ >- foreach my $disabledsubf (@disabled){ >- if ($marcitem && $marcitem->field($itemtagfield)){ >- $marcitem->field($itemtagfield)->update( $disabledsubf => "" ); >- } >- else { >- $marcitem = MARC::Record->new(); >- $marcitem->append_fields( MARC::Field->new( $itemtagfield, '', '', $disabledsubf => "" ) ); >- } >- } >- } > > my $upd_biblionumbers; > my $del_biblionumbers; >- try { >- my $schema = Koha::Database->new->schema; >- $schema->txn_do( >- sub { >- # For each item >- my $i = 1; >- foreach my $itemnumber (@itemnumbers) { >- my $item = Koha::Items->find($itemnumber); >- next >- unless $item >- ; # Should have been tested earlier, but just in case... >- my $itemdata = $item->unblessed; >- if ($del) { >+ if ( $del ) { >+ try { >+ my $schema = Koha::Database->new->schema; >+ $schema->txn_do( >+ sub { >+ foreach my $itemnumber (@itemnumbers) { >+ my $item = Koha::Items->find($itemnumber); >+ next >+ unless $item >+ ; # Should have been tested earlier, but just in case... >+ my $itemdata = $item->unblessed; > my $return = $item->safe_delete; > if ( ref( $return ) ) { > $deleted_items++; >@@ -192,100 +164,137 @@ if ($op eq "action") { > } > } > } >- else { >- my $modified_holds_priority = 0; >- if ( defined $exclude_from_local_holds_priority && $exclude_from_local_holds_priority ne "" ) { >- if(!defined $item->exclude_from_local_holds_priority || $item->exclude_from_local_holds_priority != $exclude_from_local_holds_priority) { >- $item->exclude_from_local_holds_priority($exclude_from_local_holds_priority)->store; >- $modified_holds_priority = 1; >- } >- } >- my $modified = 0; >- if ( $values_to_modify || $values_to_blank ) { >- my $localmarcitem = Item2Marc($itemdata); >+ if (@not_deleted) { >+ Koha::Exceptions::Exception->throw( >+ 'Some items have not been deleted, rolling back'); >+ } >+ } >+ ); >+ } >+ catch { >+ warn $_; >+ if ( $_->isa('Koha::Exceptions::Exception') ) { >+ $template->param( deletion_failed => 1 ); >+ } >+ die "Something terrible has happened!" >+ if ($_ =~ /Rollback failed/); # Rollback failed >+ }; >+ } > >- for ( my $i = 0 ; $i < @tags ; $i++ ) { >- my $search = $searches[$i]; >- next unless $search; >+ else { # modification > >- my $tag = $tags[$i]; >- my $subfield = $subfields[$i]; >- my $replace = $replaces[$i]; >+ my @columns = Koha::Items->columns; > >- my $value = $localmarcitem->field( $tag )->subfield( $subfield ); >- my $old_value = $value; >+ my $new_item_data; >+ my @columns_with_regex; >+ for my $c ( @columns ) { >+ if ( $c eq 'more_subfields_xml' ) { >+ my @more_subfields_xml = $input->multi_param("items.more_subfields_xml"); >+ my @unlinked_item_subfields; >+ for my $subfield ( @more_subfields_xml ) { >+ my $v = $input->param('items.more_subfields_xml_' . $subfield); >+ push @unlinked_item_subfields, $subfield, $v; >+ } >+ if ( @unlinked_item_subfields ) { >+ my $marc = MARC::Record->new(); >+ # use of tag 999 is arbitrary, and doesn't need to match the item tag >+ # used in the framework >+ $marc->append_fields(MARC::Field->new('999', ' ', ' ', @unlinked_item_subfields)); >+ $marc->encoding("UTF-8"); >+ # FIXME This is WRONG! We need to use the values that haven't been modified by the batch tool! >+ $new_item_data->{more_subfields_xml} = $marc->as_xml("USMARC"); >+ next; >+ } >+ $new_item_data->{more_subfields_xml} = undef; >+ # FIXME deal with more_subfields_xml and @subfields_to_blank >+ } elsif ( grep { $c eq $_ } @subfields_to_blank ) { >+ # Empty this column >+ $new_item_data->{$c} = undef >+ } else { > >- my @available_modifiers = qw( i g ); >- my $retained_modifiers = q||; >- for my $modifier ( split //, $modifiers[$i] ) { >- $retained_modifiers .= $modifier >- if grep {/$modifier/} @available_modifiers; >- } >- if ( $retained_modifiers =~ m/^(ig|gi)$/ ) { >- $value =~ s/$search/$replace/ig; >- } >- elsif ( $retained_modifiers eq 'i' ) { >- $value =~ s/$search/$replace/i; >- } >- elsif ( $retained_modifiers eq 'g' ) { >- $value =~ s/$search/$replace/g; >- } >- else { >- $value =~ s/$search/$replace/; >- } >+ my @v = grep { $_ ne "" } >+ uniq $input->multi_param( "items." . $c ); > >- my @fields_to = $localmarcitem->field($tag); >- foreach my $field_to_update ( @fields_to ) { >- unless ( $old_value eq $value ) { >- $modified++; >- $field_to_update->update( $subfield => $value ); >- } >- } >+ next unless @v; >+ >+ $new_item_data->{$c} = join ' | ', @v; >+ } >+ >+ if ( my $regex_search = $input->param('items.'.$c.'_regex_search') ) { >+ push @columns_with_regex, $c; >+ } >+ } >+ >+ try { >+ my $schema = Koha::Database->new->schema; >+ $schema->txn_do( >+ sub { >+ >+ foreach my $itemnumber (@itemnumbers) { >+ my $item = Koha::Items->find($itemnumber); >+ next >+ unless $item >+ ; # Should have been tested earlier, but just in case... >+ my $itemdata = $item->unblessed; >+ >+ my $modified_holds_priority = 0; >+ if ( defined $exclude_from_local_holds_priority && $exclude_from_local_holds_priority ne "" ) { >+ if(!defined $item->exclude_from_local_holds_priority || $item->exclude_from_local_holds_priority != $exclude_from_local_holds_priority) { >+ $item->exclude_from_local_holds_priority($exclude_from_local_holds_priority)->store; >+ $modified_holds_priority = 1; > } >+ } > >- $modified += UpdateMarcWith( $marcitem, $localmarcitem ); >- if ($modified) { >- eval { >- if ( >- my $item = ModItemFromMarc( >- $localmarcitem, >- $itemdata->{biblionumber}, >- $itemnumber, >- { skip_record_index => 1 }, >- ) >- ) >- { >- LostItem( >- $itemnumber, >- 'batchmod', >- undef, >- { skip_record_index => 1 } >- ) if $item->{itemlost} >- and not $itemdata->{itemlost}; >- } >- }; >- push @$upd_biblionumbers, $itemdata->{'biblionumber'}; >+ my $modified = 0; >+ for my $c ( @columns_with_regex ) { >+ my $regex_search = $input->param('items.'.$c.'_regex_search'); >+ my $old_value = $item->$c; >+ >+ my $value = apply_regex( >+ { >+ search => $regex_search, >+ replace => $input->param( >+ 'items' . $c . '_regex_replace' >+ ), >+ modifiers => $input->param( >+ 'items' . $c . '_regex_modifiers' >+ ), >+ value => $old_value, >+ } >+ ); >+ unless ( $old_value eq $value ) { >+ $modified++; >+ $item->$c($value); > } > } >+ >+ $modified += scalar(keys %$new_item_data); # FIXME This is incorrect if old value == new value. Should we loop of the keys and compare the before/after values? >+ if ( $modified) { >+ my $itemlost_pre = $item->itemlost; >+ $item->set($new_item_data)->store({skip_record_index => 1}); >+ >+ push @$upd_biblionumbers, $itemdata->{'biblionumber'}; >+ >+ LostItem( >+ $item->itemnumber, 'batchmod', undef, >+ { skip_record_index => 1 } >+ ) if $item->itemlost >+ and not $itemlost_pre; >+ } >+ > $modified_items++ if $modified || $modified_holds_priority; > $modified_fields += $modified + $modified_holds_priority; > } >- $i++; >- } >- if (@not_deleted) { >- Koha::Exceptions::Exception->throw( >- 'Some items have not been deleted, rolling back'); > } >- } >- ); >- } >- catch { >- if ( $_->isa('Koha::Exceptions::Exception') ) { >- $template->param( deletion_failed => 1 ); >+ ); > } >- die "Something terrible has happened!" >- if ($_ =~ /Rollback failed/); # Rollback failed >- }; >+ catch { >+ warn $_; >+ die "Something terrible has happened!" >+ if ($_ =~ /Rollback failed/); # Rollback failed >+ }; >+ } >+ > $upd_biblionumbers = [ uniq @$upd_biblionumbers ]; # Only update each bib once > > # Don't send specialUpdate for records we are going to delete >@@ -320,6 +329,7 @@ if ($op eq "action") { > $template->param( "job_completed" => 1 ); > } > >+ > # Calling the template > $template->param( > modified_items => $modified_items, >@@ -588,3 +598,32 @@ sub UpdateMarcWith { > } > return $modified; > } >+ >+sub apply_regex { >+ my ($params) = @_; >+ my $search = $params->{search}; >+ my $replace = $params->{replace}; >+ my $modifiers = $params->{modifiers} || []; >+ my $value = $params->{value}; >+ >+ my @available_modifiers = qw( i g ); >+ my $retained_modifiers = q||; >+ for my $modifier ( split //, @$modifiers ) { >+ $retained_modifiers .= $modifier >+ if grep { /$modifier/ } @available_modifiers; >+ } >+ if ( $retained_modifiers =~ m/^(ig|gi)$/ ) { >+ $value =~ s/$search/$replace/ig; >+ } >+ elsif ( $retained_modifiers eq 'i' ) { >+ $value =~ s/$search/$replace/i; >+ } >+ elsif ( $retained_modifiers eq 'g' ) { >+ $value =~ s/$search/$replace/g; >+ } >+ else { >+ $value =~ s/$search/$replace/; >+ } >+ >+ return $value; >+} >-- >2.25.1
<b>You cannot view the attachment while viewing its details because your browser does not support IFRAMEs. <a href="attachment.cgi?id=126328">View the attachment on a separate page</a>.</b>
View Attachment As Diff
View Attachment As Raw
Actions:
View
|
Diff
|
Splinter Review
Attachments on
bug 28445
:
121372
|
123221
|
123222
|
123223
|
123224
|
123225
|
123226
|
123227
|
123228
|
123229
|
123239
|
123240
|
125801
|
125802
|
125803
|
125804
|
125805
|
125808
|
125810
|
125811
|
125812
|
125817
|
125865
|
126046
|
126047
|
126136
|
126137
|
126138
|
126139
|
126140
|
126141
|
126142
|
126217
|
126218
|
126219
|
126223
|
126328
|
126329
|
126330
|
126331
|
126332
|
126333
|
126334
|
126335
|
126336
|
126337
|
126338
|
126339
|
126352
|
126353
|
126354
|
126355
|
126356
|
126357
|
126358
|
126359
|
126360
|
126361
|
126362
|
126363
|
126364
|
126365
|
126366
|
126367
|
126374
|
126375
|
126376
|
126377
|
126378
|
126379
|
126380
|
126381
|
126382
|
126383
|
126384
|
126385
|
126386
|
126387
|
126388
|
126389
|
127033
|
127037
|
127038
|
127039
|
128007
|
128008
|
128009