Description
Tomás Cohen Arazi
2016-05-12 13:21:33 UTC
Created attachment 51537 [details] [review] Bug 16506: Make rebuild_zebra.pl use XML as default This patch deprecates the -x switch, making XML the default serialization format used by rebuild_zebra.pl. It doesn't remove the option switch, but raises a warning for the end user about the deprecation so they fix their cronjobs. Later we could remove it. To test: - Disable all indexing (daemon/cronjob) - Create 2 records - Edit one of them, delete the other one - Verify they are queued for updates in zebraqueue - sudo koha-mysql kohadev > SELECT * FROM zebraqueue WHERE done=0 ... | 265 | 265 | specialUpdate | biblioserver | 1 | 2016-05-13 14:23:45 | | 266 | 1 | recordDelete | biblioserver | 1 | 2016-05-16 14:14:33 | | 267 | 2 | specialUpdate | biblioserver | 1 | 2016-05-16 14:15:06 | +-----+--------------------+---------------+--------------+------+---------------------+ - Now go to koha-shell $ sudo koha-shell kohadev ; cd kohaclone - Run: $ misc/migration_tools/rebuild_zebra.pl -k -b -z You will get something similar to this: NOTHING cleaned : the export /tmp/jI0OeHy6Tn has been kept. You can re-run this script with the -s and -d /tmp/jI0OeHy6Tn parameters if you just want to rebuild zebra after changing the record.abs or another zebra config file - Verify * less /tmp/jI0OeHy6Tn/del_biblio/exported_records * less /tmp/jI0OeHy6Tn/upd_biblio/exported_records => FAIL: They contain the records you added/modified/deleted but they are in USMARC format - Apply the patch - Mark your records for indexing (in koha-mysql kohadev) > UPDATE zebraqueue SET done=0 WHERE id > 264 - Run: $ misc/migration_tools/rebuild_zebra.pl -k -b -z You will get something similar to this: <WARNINGS> [1] NOTHING cleaned : the export /tmp/jI0OeHy6Tn has been kept. You can re-run this script with the -s and -d /tmp/jI0OeHy6Tn parameters if you just want to rebuild zebra after changing the record.abs or another zebra config file - Verify * less /tmp/jI0OeHy6Tn/del_biblio/exported_records * less /tmp/jI0OeHy6Tn/upd_biblio/exported_records => SUCCESS: Data is correctly in XML format - Run: $ misc/migration_tools/rebuild_zebra.pl -k -b -z -noxml You will get something similar to this: <WARNINGS> [1] NOTHING cleaned : the export /tmp/jI0OeHy6Tn has been kept. You can re-run this script with the -s and -d /tmp/jI0OeHy6Tn parameters if you just want to rebuild zebra after changing the record.abs or another zebra config file - Verify * less /tmp/jI0OeHy6Tn/del_biblio/exported_records * less /tmp/jI0OeHy6Tn/upd_biblio/exported_records => SUCCESS: Data is correctly in USMARC format - Sign off :-D [1] Warnings covered by a followup Created attachment 51538 [details] [review] Bug 16506: (Followup) remove warnings Created attachment 51552 [details] [review] Bug 16506: (followup) Fix wrong option switch warning message Created attachment 51566 [details] [review] [SIGNED-OFF] Bug 16506: Make rebuild_zebra.pl use XML as default This patch deprecates the -x switch, making XML the default serialization format used by rebuild_zebra.pl. It doesn't remove the option switch, but raises a warning for the end user about the deprecation so they fix their cronjobs. Later we could remove it. To test: - Disable all indexing (daemon/cronjob) - Create 2 records - Edit one of them, delete the other one - Verify they are queued for updates in zebraqueue - sudo koha-mysql kohadev > SELECT * FROM zebraqueue WHERE done=0 ... | 265 | 265 | specialUpdate | biblioserver | 1 | 2016-05-13 14:23:45 | | 266 | 1 | recordDelete | biblioserver | 1 | 2016-05-16 14:14:33 | | 267 | 2 | specialUpdate | biblioserver | 1 | 2016-05-16 14:15:06 | +-----+--------------------+---------------+--------------+------+---------------------+ - Now go to koha-shell $ sudo koha-shell kohadev ; cd kohaclone - Run: $ misc/migration_tools/rebuild_zebra.pl -k -b -z You will get something similar to this: NOTHING cleaned : the export /tmp/jI0OeHy6Tn has been kept. You can re-run this script with the -s and -d /tmp/jI0OeHy6Tn parameters if you just want to rebuild zebra after changing the record.abs or another zebra config file - Verify * less /tmp/jI0OeHy6Tn/del_biblio/exported_records * less /tmp/jI0OeHy6Tn/upd_biblio/exported_records => FAIL: They contain the records you added/modified/deleted but they are in USMARC format - Apply the patch - Mark your records for indexing (in koha-mysql kohadev) > UPDATE zebraqueue SET done=0 WHERE id > 264 - Run: $ misc/migration_tools/rebuild_zebra.pl -k -b -z You will get something similar to this: <WARNINGS> [1] NOTHING cleaned : the export /tmp/jI0OeHy6Tn has been kept. You can re-run this script with the -s and -d /tmp/jI0OeHy6Tn parameters if you just want to rebuild zebra after changing the record.abs or another zebra config file - Verify * less /tmp/jI0OeHy6Tn/del_biblio/exported_records * less /tmp/jI0OeHy6Tn/upd_biblio/exported_records => SUCCESS: Data is correctly in XML format - Run: $ misc/migration_tools/rebuild_zebra.pl -k -b -z -noxml You will get something similar to this: <WARNINGS> [1] NOTHING cleaned : the export /tmp/jI0OeHy6Tn has been kept. You can re-run this script with the -s and -d /tmp/jI0OeHy6Tn parameters if you just want to rebuild zebra after changing the record.abs or another zebra config file - Verify * less /tmp/jI0OeHy6Tn/del_biblio/exported_records * less /tmp/jI0OeHy6Tn/upd_biblio/exported_records => SUCCESS: Data is correctly in USMARC format - Sign off :-D [1] Warnings covered by a followup Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> On top of Bug 16505 Work as described following test plan, usmarc default pre patch, post patch xml default and usmarc on request. No errors (all patchset) Created attachment 51567 [details] [review] [SIGNED-OFF] Bug 16506: (Followup) remove warnings Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> Created attachment 51568 [details] [review] [SIGNED-OFF] Bug 16506: (followup) Fix wrong option switch warning message Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> Created attachment 51675 [details] [review] Bug 16506: Make rebuild_zebra.pl use XML as default This patch deprecates the -x switch, making XML the default serialization format used by rebuild_zebra.pl. It doesn't remove the option switch, but raises a warning for the end user about the deprecation so they fix their cronjobs. Later we could remove it. To test: - Disable all indexing (daemon/cronjob) - Create 2 records - Edit one of them, delete the other one - Verify they are queued for updates in zebraqueue - sudo koha-mysql kohadev > SELECT * FROM zebraqueue WHERE done=0 ... | 265 | 265 | specialUpdate | biblioserver | 1 | 2016-05-13 14:23:45 | | 266 | 1 | recordDelete | biblioserver | 1 | 2016-05-16 14:14:33 | | 267 | 2 | specialUpdate | biblioserver | 1 | 2016-05-16 14:15:06 | +-----+--------------------+---------------+--------------+------+---------------------+ - Now go to koha-shell $ sudo koha-shell kohadev ; cd kohaclone - Run: $ misc/migration_tools/rebuild_zebra.pl -k -b -z You will get something similar to this: NOTHING cleaned : the export /tmp/jI0OeHy6Tn has been kept. You can re-run this script with the -s and -d /tmp/jI0OeHy6Tn parameters if you just want to rebuild zebra after changing the record.abs or another zebra config file - Verify * less /tmp/jI0OeHy6Tn/del_biblio/exported_records * less /tmp/jI0OeHy6Tn/upd_biblio/exported_records => FAIL: They contain the records you added/modified/deleted but they are in USMARC format - Apply the patch - Mark your records for indexing (in koha-mysql kohadev) > UPDATE zebraqueue SET done=0 WHERE id > 264 - Run: $ misc/migration_tools/rebuild_zebra.pl -k -b -z You will get something similar to this: <WARNINGS> [1] NOTHING cleaned : the export /tmp/jI0OeHy6Tn has been kept. You can re-run this script with the -s and -d /tmp/jI0OeHy6Tn parameters if you just want to rebuild zebra after changing the record.abs or another zebra config file - Verify * less /tmp/jI0OeHy6Tn/del_biblio/exported_records * less /tmp/jI0OeHy6Tn/upd_biblio/exported_records => SUCCESS: Data is correctly in XML format - Run: $ misc/migration_tools/rebuild_zebra.pl -k -b -z -noxml You will get something similar to this: <WARNINGS> [1] NOTHING cleaned : the export /tmp/jI0OeHy6Tn has been kept. You can re-run this script with the -s and -d /tmp/jI0OeHy6Tn parameters if you just want to rebuild zebra after changing the record.abs or another zebra config file - Verify * less /tmp/jI0OeHy6Tn/del_biblio/exported_records * less /tmp/jI0OeHy6Tn/upd_biblio/exported_records => SUCCESS: Data is correctly in USMARC format - Sign off :-D [1] Warnings covered by a followup Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> On top of Bug 16505 Work as described following test plan, usmarc default pre patch, post patch xml default and usmarc on request. No errors (all patchset) Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org> Created attachment 51676 [details] [review] Bug 16506: (Followup) remove warnings Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org> Created attachment 51677 [details] [review] Bug 16506: (followup) Fix wrong option switch warning message Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org> Created attachment 51678 [details] [review] Bug 16506: Remove warning for UNIMARC installs Use of uninitialized value in numeric eq (==) Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org> Pushed to master for Koha 16.05. Thanks Tomas, Jonathan! It doesn't look like a bugfix (severity = normal), can someone confirm this is an enhancement ? (In reply to Julian Maurice from comment #12) > It doesn't look like a bugfix (severity = normal), can someone confirm this > is an enhancement ? Well, it might be. Because bug 11096 introduced XML as exchange format for DOM setups, to allow bigger records to get indexed, and we then deprecated GRS-1. To me this is something we just forgot to tweak after 11096. But it is really up to you to include it in 3.22 as it supposes a behaviour change for people using the cronjob to update their indexes. |