Bug 11399 - batch rebuild items tables
Summary: batch rebuild items tables
Status: CLOSED FIXED
Alias: None
Product: Koha
Classification: Unclassified
Component: Command-line Utilities (show other bugs)
Version: master
Hardware: All All
: P5 - low enhancement (vote)
Assignee: stephane.delaune
QA Contact: Testopia
URL:
Whiteboard:
Keywords:
Depends on:
Blocks: 13023
  Show dependency treegraph
 
Reported: 2013-12-16 13:10 UTC by stephane.delaune
Modified: 2016-06-21 21:39 UTC (History)
4 users (show)

See Also:
Change sponsored?: ---
Patch complexity: ---
Bot Control: ---
When did the bot last check this:
Who signed the patch off:
Text to go in the release notes:


Attachments
proposed patch (4.77 KB, patch)
2013-12-16 13:20 UTC, stephane.delaune
Details | Diff | Splinter Review
proposed patch (5.30 KB, patch)
2013-12-17 11:50 UTC, stephane.delaune
Details | Diff | Splinter Review
[SIGNED-OFF] Bug 11399: batch Rebuild Items Tables (6.05 KB, patch)
2014-06-23 01:02 UTC, Bernardo Gonzalez Kriegel
Details | Diff | Splinter Review
followup : QA fixes (1.37 KB, patch)
2014-09-11 08:42 UTC, stephane.delaune
Details | Diff | Splinter Review
[PASSED QA] Bug 11399: batch Rebuild Items Tables (6.12 KB, patch)
2014-10-17 14:52 UTC, Kyle M Hall
Details | Diff | Splinter Review
[PASSED QA] Bug 11399: QA fixes (1.43 KB, patch)
2014-10-17 14:52 UTC, Kyle M Hall
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description stephane.delaune 2013-12-16 13:10:39 UTC
This script rebuilds the non-MARC DB items table from the MARC values (eg. in 995 for unimarc, 952 in marc21). You can/must use it when you change your mapping.

This script is similar to batchRebuildBiblioTables(which works on biblio an biblioitems table), but works on items only.
Comment 1 stephane.delaune 2013-12-16 13:20:08 UTC Comment hidden (obsolete)
Comment 2 Katrin Fischer 2013-12-17 09:15:24 UTC
Hi Stephane, I am a bit confused about the use of this, because I think we don't keep the item information in the MARC any more?
Comment 3 stephane.delaune 2013-12-17 09:48:31 UTC
(In reply to Katrin Fischer from comment #2)
> Hi Stephane, I am a bit confused about the use of this, because I think we
> don't keep the item information in the MARC any more?

Hi Katrin, in fact, informations about items aren't stored directly in MARC. However, when editing a record, MARC items informations can be regenerated temporarily (that's what serves '1' when we call the function 'GetMarcBiblio($biblionumber,1)'). When a mapping is changed in "MARC bibliographic framework" or "Koha to MARC mapping" (admin/koha2marclinks.pl on items), it does not change the existing items. For example in UNIMARC, without this patch, if I want to link the 995$j with items.stocknumber, this information will remain in items.more_subfields_xml for old items as long as they have not been modified manually. This script forces rewriting of items taking new mapping into account (in our example, all 995$j will then be moved from items.more_subfields_xml to items.stocknumber).
Comment 4 Katrin Fischer 2013-12-17 10:52:19 UTC
Hi Stephane, that makes sense - maybe we should change the bug description? I think this will only be needed if you change the mapping from a before unmapped field or when you map an unmapped one (more_subfields_xml)
Comment 5 stephane.delaune 2013-12-17 11:50:56 UTC Comment hidden (obsolete)
Comment 6 Bernardo Gonzalez Kriegel 2014-06-23 01:02:48 UTC Comment hidden (obsolete)
Comment 7 Katrin Fischer 2014-07-06 20:43:25 UTC
Comment on attachment 29156 [details] [review]
[SIGNED-OFF] Bug 11399: batch Rebuild Items Tables

Review of attachment 29156 [details] [review]:
-----------------------------------------------------------------

2 small things from code review:

::: misc/batchRebuildItemsTables.pl
@@ +1,4 @@
> +#!/usr/bin/perl
> +use 5.10.0;
> +use strict;
> +use warnings;

Please switch to: use Modern::Perl; :)

@@ +94,5 @@
> +    }
> +}
> +
> +$dbh->do("UPDATE systempreferences SET value=$CataloguingLog WHERE variable='CataloguingLog'");
> +$dbh->do("UPDATE systempreferences SET value=$dontmerge where variable='dontmerge'");

Please switch to using placeholders instead of variables.
Comment 8 stephane.delaune 2014-09-11 08:42:53 UTC Comment hidden (obsolete)
Comment 9 Kyle M Hall 2014-10-17 14:52:03 UTC
Created attachment 32511 [details] [review]
[PASSED QA] Bug 11399: batch Rebuild Items Tables

You can/must use it when you change items mapping.
this script rebuilds the non-MARC DB items table from the MARC values

usage : batchRebuildItemsTables.pl [ -h ][ -c ][ -t ][ -where ]

Options:
   -h --help       (or without arguments) shows this help message
   -c              Confirm: rebuild non marc DB (may be long)
   -t              test only, change nothing in DB
   --where         add where condition on default query
                   (eg. -where 'biblio.biblionumber<100')

(copied test plan)
to test it :

create a test item (eg on biblionumber 3) with a non mapped subfield with a test value.
check real items DB row ( SELECT * from items where biblionumber=3 ) => non mapped value is in more_subfields_xml
map the non mapped subfield with unused items column
re-check in Db that nothing changes
run ./batchRebuildItemsTables.pl -c --where 'biblio.biblionumber=3'
check real items DB row ( SELECT * from items where biblionumber=3 ) => new mapped value is not in more_subfields_xml anymore but in the mapped column

Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Work as described, no koha-qa errors.
Add a new item to record, save with value in unmapped subfield ('x')
Change mapping, run script, value on newly mapped column

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Comment 10 Kyle M Hall 2014-10-17 14:52:10 UTC
Created attachment 32512 [details] [review]
[PASSED QA] Bug 11399: QA fixes

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Comment 11 Kyle M Hall 2014-10-17 14:53:35 UTC
Note, I found it a bit confusing that a test run required both -t *and* -c. I would expect each to be independent, and use of both would be the same as -t alone. However, we have no formal guidelines for this yet, so I do not feel it is proper to fail qa on this point.
Comment 12 Tomás Cohen Arazi 2015-04-23 16:11:44 UTC
Patches pushed to master.

Thanks Stephane!