Bug 16506

Summary: rebuild_zebra.pl still using USMARC as default
Product: Koha Reporter: Tomás Cohen Arazi <tomascohen>
Component: Architecture, internals, and plumbingAssignee: Tomás Cohen Arazi <tomascohen>
Status: CLOSED FIXED QA Contact: Testopia <testopia>
Severity: normal    
Priority: P5 - low CC: abl, bgkriegel, jonathan.druart, julian.maurice, kyle
Version: master   
Hardware: All   
OS: All   
Change sponsored?: --- Patch complexity: ---
Documentation contact: Documentation submission:
Text to go in the release notes:
Version(s) released in:
Bug Depends on: 16505    
Bug Blocks: 12641, 16830    
Attachments: Bug 16506: Make rebuild_zebra.pl use XML as default
Bug 16506: (Followup) remove warnings
Bug 16506: (followup) Fix wrong option switch warning message
[SIGNED-OFF] Bug 16506: Make rebuild_zebra.pl use XML as default
[SIGNED-OFF] Bug 16506: (Followup) remove warnings
[SIGNED-OFF] Bug 16506: (followup) Fix wrong option switch warning message
Bug 16506: Make rebuild_zebra.pl use XML as default
Bug 16506: (Followup) remove warnings
Bug 16506: (followup) Fix wrong option switch warning message
Bug 16506: Remove warning for UNIMARC installs

Description Tomás Cohen Arazi 2016-05-12 13:21:33 UTC
The current rebuild_zebra.pl still uses usmarc as serialization format.
Comment 1 Tomás Cohen Arazi 2016-05-16 15:19:32 UTC Comment hidden (obsolete)
Comment 2 Tomás Cohen Arazi 2016-05-16 15:19:37 UTC Comment hidden (obsolete)
Comment 3 Tomás Cohen Arazi 2016-05-16 19:09:43 UTC Comment hidden (obsolete)
Comment 4 Bernardo Gonzalez Kriegel 2016-05-17 11:16:42 UTC Comment hidden (obsolete)
Comment 5 Bernardo Gonzalez Kriegel 2016-05-17 11:16:50 UTC Comment hidden (obsolete)
Comment 6 Bernardo Gonzalez Kriegel 2016-05-17 11:16:58 UTC Comment hidden (obsolete)
Comment 7 Jonathan Druart 2016-05-21 09:22:47 UTC
Created attachment 51675 [details] [review]
Bug 16506: Make rebuild_zebra.pl use XML as default

This patch deprecates the -x switch, making XML the default serialization format
used by rebuild_zebra.pl. It doesn't remove the option switch, but raises a warning
for the end user about the deprecation so they fix their cronjobs. Later we could remove it.

To test:
- Disable all indexing (daemon/cronjob)
- Create 2 records
- Edit one of them, delete the other one
- Verify they are queued for updates in zebraqueue
- sudo koha-mysql kohadev
  > SELECT * FROM zebraqueue WHERE done=0
...
| 265 |                265 | specialUpdate | biblioserver |    1 | 2016-05-13 14:23:45 |
| 266 |                  1 | recordDelete  | biblioserver |    1 | 2016-05-16 14:14:33 |
| 267 |                  2 | specialUpdate | biblioserver |    1 | 2016-05-16 14:15:06 |
+-----+--------------------+---------------+--------------+------+---------------------+
- Now go to koha-shell
  $ sudo koha-shell kohadev ; cd kohaclone
- Run:
  $ misc/migration_tools/rebuild_zebra.pl -k -b -z

  You will get something similar to this:
NOTHING cleaned : the export /tmp/jI0OeHy6Tn has been kept.
You can re-run this script with the -s  and -d /tmp/jI0OeHy6Tn parameters
if you just want to rebuild zebra after changing the record.abs
or another zebra config file
- Verify
  * less /tmp/jI0OeHy6Tn/del_biblio/exported_records
  * less /tmp/jI0OeHy6Tn/upd_biblio/exported_records
=> FAIL: They contain the records you added/modified/deleted but they are in
         USMARC format
- Apply the patch
- Mark your records for indexing (in koha-mysql kohadev)
  > UPDATE zebraqueue SET done=0 WHERE id > 264
- Run:
  $ misc/migration_tools/rebuild_zebra.pl -k -b -z

  You will get something similar to this:
<WARNINGS> [1]
NOTHING cleaned : the export /tmp/jI0OeHy6Tn has been kept.
You can re-run this script with the -s  and -d /tmp/jI0OeHy6Tn parameters
if you just want to rebuild zebra after changing the record.abs
or another zebra config file
- Verify
  * less /tmp/jI0OeHy6Tn/del_biblio/exported_records
  * less /tmp/jI0OeHy6Tn/upd_biblio/exported_records
=> SUCCESS: Data is correctly in XML format
- Run:
  $ misc/migration_tools/rebuild_zebra.pl -k -b -z -noxml

  You will get something similar to this:
<WARNINGS> [1]
NOTHING cleaned : the export /tmp/jI0OeHy6Tn has been kept.
You can re-run this script with the -s  and -d /tmp/jI0OeHy6Tn parameters
if you just want to rebuild zebra after changing the record.abs
or another zebra config file
- Verify
  * less /tmp/jI0OeHy6Tn/del_biblio/exported_records
  * less /tmp/jI0OeHy6Tn/upd_biblio/exported_records
=> SUCCESS: Data is correctly in USMARC format
- Sign off :-D

[1] Warnings covered by a followup

Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
On top of Bug 16505
Work as described following test plan, usmarc default pre patch,
post patch xml default and usmarc on request.
No errors (all patchset)

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Comment 8 Jonathan Druart 2016-05-21 09:22:50 UTC
Created attachment 51676 [details] [review]
Bug 16506: (Followup) remove warnings

Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Comment 9 Jonathan Druart 2016-05-21 09:22:54 UTC
Created attachment 51677 [details] [review]
Bug 16506: (followup) Fix wrong option switch warning message

Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Comment 10 Jonathan Druart 2016-05-21 09:23:00 UTC
Created attachment 51678 [details] [review]
Bug 16506: Remove warning for UNIMARC installs

Use of uninitialized value in numeric eq (==)

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Comment 11 Kyle M Hall 2016-05-23 17:30:43 UTC
Pushed to master for Koha 16.05. Thanks Tomas, Jonathan!
Comment 12 Julian Maurice 2016-05-25 09:25:00 UTC
It doesn't look like a bugfix (severity = normal), can someone confirm this is an enhancement ?
Comment 13 Tomás Cohen Arazi 2016-05-25 13:51:24 UTC
(In reply to Julian Maurice from comment #12)
> It doesn't look like a bugfix (severity = normal), can someone confirm this
> is an enhancement ?

Well, it might be. Because bug 11096 introduced XML as exchange format for DOM setups, to allow bigger records to get indexed, and we then deprecated GRS-1. To me this is something we just forgot to tweak after 11096. But it is really up to you to include it in 3.22 as it supposes a behaviour change for people using the cronjob to update their indexes.