Bug 16506 - rebuild_zebra.pl still using USMARC as default
Summary: rebuild_zebra.pl still using USMARC as default
Status: CLOSED FIXED
Alias: None
Product: Koha
Classification: Unclassified
Component: Architecture, internals, and plumbing (show other bugs)
Version: master
Hardware: All All
: P5 - low normal (vote)
Assignee: Tomás Cohen Arazi
QA Contact: Testopia
URL:
Keywords:
Depends on: 16505
Blocks: 12641 16830
  Show dependency treegraph
 
Reported: 2016-05-12 13:21 UTC by Tomás Cohen Arazi
Modified: 2017-12-07 22:16 UTC (History)
5 users (show)

See Also:
Change sponsored?: ---
Patch complexity: ---
Text to go in the release notes:
Version(s) released in:


Attachments
Bug 16506: Make rebuild_zebra.pl use XML as default (15.87 KB, patch)
2016-05-16 15:19 UTC, Tomás Cohen Arazi
Details | Diff | Splinter Review
Bug 16506: (Followup) remove warnings (1.53 KB, patch)
2016-05-16 15:19 UTC, Tomás Cohen Arazi
Details | Diff | Splinter Review
Bug 16506: (followup) Fix wrong option switch warning message (870 bytes, patch)
2016-05-16 19:09 UTC, Tomás Cohen Arazi
Details | Diff | Splinter Review
[SIGNED-OFF] Bug 16506: Make rebuild_zebra.pl use XML as default (16.10 KB, patch)
2016-05-17 11:16 UTC, Bernardo Gonzalez Kriegel
Details | Diff | Splinter Review
[SIGNED-OFF] Bug 16506: (Followup) remove warnings (1.60 KB, patch)
2016-05-17 11:16 UTC, Bernardo Gonzalez Kriegel
Details | Diff | Splinter Review
[SIGNED-OFF] Bug 16506: (followup) Fix wrong option switch warning message (947 bytes, patch)
2016-05-17 11:16 UTC, Bernardo Gonzalez Kriegel
Details | Diff | Splinter Review
Bug 16506: Make rebuild_zebra.pl use XML as default (16.15 KB, patch)
2016-05-21 09:22 UTC, Jonathan Druart
Details | Diff | Splinter Review
Bug 16506: (Followup) remove warnings (1.66 KB, patch)
2016-05-21 09:22 UTC, Jonathan Druart
Details | Diff | Splinter Review
Bug 16506: (followup) Fix wrong option switch warning message (1007 bytes, patch)
2016-05-21 09:22 UTC, Jonathan Druart
Details | Diff | Splinter Review
Bug 16506: Remove warning for UNIMARC installs (1.47 KB, patch)
2016-05-21 09:23 UTC, Jonathan Druart
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Tomás Cohen Arazi 2016-05-12 13:21:33 UTC
The current rebuild_zebra.pl still uses usmarc as serialization format.
Comment 1 Tomás Cohen Arazi 2016-05-16 15:19:32 UTC Comment hidden (obsolete)
Comment 2 Tomás Cohen Arazi 2016-05-16 15:19:37 UTC Comment hidden (obsolete)
Comment 3 Tomás Cohen Arazi 2016-05-16 19:09:43 UTC Comment hidden (obsolete)
Comment 4 Bernardo Gonzalez Kriegel 2016-05-17 11:16:42 UTC Comment hidden (obsolete)
Comment 5 Bernardo Gonzalez Kriegel 2016-05-17 11:16:50 UTC Comment hidden (obsolete)
Comment 6 Bernardo Gonzalez Kriegel 2016-05-17 11:16:58 UTC Comment hidden (obsolete)
Comment 7 Jonathan Druart 2016-05-21 09:22:47 UTC
Created attachment 51675 [details] [review]
Bug 16506: Make rebuild_zebra.pl use XML as default

This patch deprecates the -x switch, making XML the default serialization format
used by rebuild_zebra.pl. It doesn't remove the option switch, but raises a warning
for the end user about the deprecation so they fix their cronjobs. Later we could remove it.

To test:
- Disable all indexing (daemon/cronjob)
- Create 2 records
- Edit one of them, delete the other one
- Verify they are queued for updates in zebraqueue
- sudo koha-mysql kohadev
  > SELECT * FROM zebraqueue WHERE done=0
...
| 265 |                265 | specialUpdate | biblioserver |    1 | 2016-05-13 14:23:45 |
| 266 |                  1 | recordDelete  | biblioserver |    1 | 2016-05-16 14:14:33 |
| 267 |                  2 | specialUpdate | biblioserver |    1 | 2016-05-16 14:15:06 |
+-----+--------------------+---------------+--------------+------+---------------------+
- Now go to koha-shell
  $ sudo koha-shell kohadev ; cd kohaclone
- Run:
  $ misc/migration_tools/rebuild_zebra.pl -k -b -z

  You will get something similar to this:
NOTHING cleaned : the export /tmp/jI0OeHy6Tn has been kept.
You can re-run this script with the -s  and -d /tmp/jI0OeHy6Tn parameters
if you just want to rebuild zebra after changing the record.abs
or another zebra config file
- Verify
  * less /tmp/jI0OeHy6Tn/del_biblio/exported_records
  * less /tmp/jI0OeHy6Tn/upd_biblio/exported_records
=> FAIL: They contain the records you added/modified/deleted but they are in
         USMARC format
- Apply the patch
- Mark your records for indexing (in koha-mysql kohadev)
  > UPDATE zebraqueue SET done=0 WHERE id > 264
- Run:
  $ misc/migration_tools/rebuild_zebra.pl -k -b -z

  You will get something similar to this:
<WARNINGS> [1]
NOTHING cleaned : the export /tmp/jI0OeHy6Tn has been kept.
You can re-run this script with the -s  and -d /tmp/jI0OeHy6Tn parameters
if you just want to rebuild zebra after changing the record.abs
or another zebra config file
- Verify
  * less /tmp/jI0OeHy6Tn/del_biblio/exported_records
  * less /tmp/jI0OeHy6Tn/upd_biblio/exported_records
=> SUCCESS: Data is correctly in XML format
- Run:
  $ misc/migration_tools/rebuild_zebra.pl -k -b -z -noxml

  You will get something similar to this:
<WARNINGS> [1]
NOTHING cleaned : the export /tmp/jI0OeHy6Tn has been kept.
You can re-run this script with the -s  and -d /tmp/jI0OeHy6Tn parameters
if you just want to rebuild zebra after changing the record.abs
or another zebra config file
- Verify
  * less /tmp/jI0OeHy6Tn/del_biblio/exported_records
  * less /tmp/jI0OeHy6Tn/upd_biblio/exported_records
=> SUCCESS: Data is correctly in USMARC format
- Sign off :-D

[1] Warnings covered by a followup

Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
On top of Bug 16505
Work as described following test plan, usmarc default pre patch,
post patch xml default and usmarc on request.
No errors (all patchset)

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Comment 8 Jonathan Druart 2016-05-21 09:22:50 UTC
Created attachment 51676 [details] [review]
Bug 16506: (Followup) remove warnings

Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Comment 9 Jonathan Druart 2016-05-21 09:22:54 UTC
Created attachment 51677 [details] [review]
Bug 16506: (followup) Fix wrong option switch warning message

Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Comment 10 Jonathan Druart 2016-05-21 09:23:00 UTC
Created attachment 51678 [details] [review]
Bug 16506: Remove warning for UNIMARC installs

Use of uninitialized value in numeric eq (==)

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Comment 11 Kyle M Hall 2016-05-23 17:30:43 UTC
Pushed to master for Koha 16.05. Thanks Tomas, Jonathan!
Comment 12 Julian Maurice 2016-05-25 09:25:00 UTC
It doesn't look like a bugfix (severity = normal), can someone confirm this is an enhancement ?
Comment 13 Tomás Cohen Arazi 2016-05-25 13:51:24 UTC
(In reply to Julian Maurice from comment #12)
> It doesn't look like a bugfix (severity = normal), can someone confirm this
> is an enhancement ?

Well, it might be. Because bug 11096 introduced XML as exchange format for DOM setups, to allow bigger records to get indexed, and we then deprecated GRS-1. To me this is something we just forgot to tweak after 11096. But it is really up to you to include it in 3.22 as it supposes a behaviour change for people using the cronjob to update their indexes.