From cbfaa105e157d34f30c180c8c732f65d111dc558 Mon Sep 17 00:00:00 2001 From: Olli-Antti Kivilahti Date: Tue, 3 Feb 2015 13:52:26 +0200 Subject: [PATCH] Bug 13660 - rebuild_zebra_sliced.sh - Exclude export phase and use existing exported MARCXML. When looking for a bad MARC Record using the rebuild_zebra_sliced.sh, it is useful to skip the complete MARCXML exporting from Koha and reuse the exported files for Zebra indexing. This patch adds a new parameter: -x | --exclude-export Do not export Biblios from Koha, but use the existing export-dir Which depends on the: -d | --export-dir Where rebuild_zebra.pl will export data Default: $EXPORTDIR !---------! ! TEST PLAN ! !---------! 1. Run "./rebuild_zebra_sliced.sh --length 1000" to export 1000 MARC Records and slice them to one big 1000-Record chunk. 2. Realize that you get an imaginary "stack smashing detected"-error crashing your indexing at some Record you dont know of and can't make out from the indexing logging. 3. Start looking for the bad Record by running: "./rebuild_zebra_sliced.sh --exlude-export --chunk-size 10" To skip Biblios export from Koha which takes ~2h and get straight into splitting your exported biblios to chunks of 10, and indexing them. You know which chunk fails so it is much easier to find the issue there. --- misc/migration_tools/rebuild_zebra_sliced.sh | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-) diff --git a/misc/migration_tools/rebuild_zebra_sliced.sh b/misc/migration_tools/rebuild_zebra_sliced.sh index 7ddf32e..1eeb551 100755 --- a/misc/migration_tools/rebuild_zebra_sliced.sh +++ b/misc/migration_tools/rebuild_zebra_sliced.sh @@ -24,6 +24,8 @@ $scriptname -h Default: $CHUNKSSIZE -d | --export-dir Where rebuild_zebra.pl will export data Default: $EXPORTDIR + -x | --exclude-export Do not export Biblios from Koha, but use the existing + export-dir -L | --log-dir Log directory Default: $LOGDIR -r | --remove-logs Clean log directory before start @@ -110,6 +112,7 @@ OFFSET=0 LENGTH= CHUNKSSIZE=10000 EXPORTDIR=/tmp/rebuild/export +EXCLUDEEXPORT=no LOGDIR=/tmp/rebuild/logs RMLOGS=no NOCONFIRM=no @@ -140,6 +143,9 @@ while [ $1 ]; do shift LOGDIR=$1 ;; + -x | --exclude-export ) + EXCLUDEEXPORT=yes + ;; -r | --remove-logs ) RMLOGS=yes ;; @@ -235,6 +241,7 @@ echo "Start at offset: $OFFSET" echo "Total number of records to index: $LENGTH" echo "Initial chunk size: $CHUNKSSIZE" echo "Export directory: $EXPORTDIR" +echo "Exclude re-exporting: $EXCLUDEEXPORT" echo "Log directory: $LOGDIR" echo "Remove logs before start? $RMLOGS" echo "Type of record: $TYPE" @@ -274,9 +281,11 @@ if [ $RMLOGS = "yes" ]; then rm -f $LOGDIR/*.log fi -REBUILDZEBRA_CMD="$REBUILDZEBRA $TYPESWITCH -v -x -k -d $EXPORTDIR --offset $OFFSET --length $LENGTH --skip-index" -echo "\n$REBUILDZEBRA_CMD" -$REBUILDZEBRA_CMD +if [ $EXCLUDEEXPORT = "no" ]; then + REBUILDZEBRA_CMD="$REBUILDZEBRA $TYPESWITCH -v -x -k -d $EXPORTDIR --offset $OFFSET --length $LENGTH --skip-index" + echo "\n$REBUILDZEBRA_CMD" + $REBUILDZEBRA_CMD +fi EXPORTFILE= case $TYPE in -- 1.7.9.5