Bugzilla – Attachment 21097 Details for
Bug 6435
[ENH] Added daemon mode parameters to rebuild_zebra.pl
Home
|
New
|
Browse
|
Search
|
[?]
|
Reports
|
Help
|
New Account
|
Log In
[x]
|
Forgot Password
Login:
[x]
[patch]
Proper GIT patch for the daemon mode addition to rebuild_zebra.pl
0001-Bug-6435-Add-daemon-mode-to-rebuild_zebra.pl.patch (text/plain), 4.63 KB, created by
Doug Kingston
on 2013-09-13 18:43:46 UTC
(
hide
)
Description:
Proper GIT patch for the daemon mode addition to rebuild_zebra.pl
Filename:
MIME Type:
Creator:
Doug Kingston
Created:
2013-09-13 18:43:46 UTC
Size:
4.63 KB
patch
obsolete
>From 86b4a75624693264026c9b9509247d8d7194e2f5 Mon Sep 17 00:00:00 2001 >From: Doug Kingston <dpk@randomnotes.org> >Date: Fri, 13 Sep 2013 11:07:29 -0700 >Subject: [PATCH] Bug 6435 Add daemon mode to rebuild_zebra.pl > >This change adds code to check the zebraqueue table with a cheap SQL query >and a daemon loop that checks for new entries and processes them incremantally >before sleeping for a controllable number of seconds. The default is 5 seconds >which provides a near realtime search index update. This is desirable particularly >for libraries that are doing active catalogue updating. > >Help text updated. Tested against a live 3.12 system. >--- > misc/migration_tools/rebuild_zebra.pl | 54 +++++++++++++++++++++++++++------ > 1 file changed, 45 insertions(+), 9 deletions(-) > >diff --git a/misc/migration_tools/rebuild_zebra.pl b/misc/migration_tools/rebuild_zebra.pl >index a46526d..2e01e9c 100755 >--- a/misc/migration_tools/rebuild_zebra.pl >+++ b/misc/migration_tools/rebuild_zebra.pl >@@ -21,6 +21,8 @@ $|=1; # flushes output > # If the cron job starts us in an unreadable dir, we will break without > # this. > chdir $ENV{HOME} if (!(-r '.')); >+my $daemon_mode; >+my $daemon_sleep = 5; > my $directory; > my $nosanitize; > my $skip_export; >@@ -45,6 +47,8 @@ my $run_user = (getpwuid($<))[0]; > my $verbose_logging = 0; > my $zebraidx_log_opt = " -v none,fatal,warn "; > my $result = GetOptions( >+ 'daemon' => \$daemon_mode, >+ 'sleep:i' => \$daemon_sleep, > 'd:s' => \$directory, > 'r|reset' => \$reset, > 's' => \$skip_export, >@@ -152,16 +156,13 @@ if ($do_munge) { > > my $tester = XML::LibXML->new(); > >-if ($authorities) { >- index_records('authority', $directory, $skip_export, $skip_index, $process_zebraqueue, $as_xml, $noxml, $nosanitize, $do_not_clear_zebraqueue, $verbose_logging, $zebraidx_log_opt, $authorityserverdir); >-} else { >- print "skipping authorities\n" if ( $verbose_logging ); >-} >- >-if ($biblios) { >- index_records('biblio', $directory, $skip_export, $skip_index, $process_zebraqueue, $as_xml, $noxml, $nosanitize, $do_not_clear_zebraqueue, $verbose_logging, $zebraidx_log_opt, $biblioserverdir); >+if ($daemon_mode) { >+ while (1) { >+ do_one_pass() if ( zebraqueue_not_empty() ); >+ sleep $daemon_sleep; >+ } > } else { >- print "skipping biblios\n" if ( $verbose_logging ); >+ do_one_pass(); > } > > >@@ -191,6 +192,29 @@ if ($keep_export) { > } > } > >+sub do_one_pass { >+ if ($authorities) { >+ index_records('authority', $directory, $skip_export, $skip_index, $process_zebraqueue, $as_xml, $noxml, $nosanitize, $do_not_clear_zebraqueue, $verbose_logging, $zebraidx_log_opt, $authorityserverdir); >+ } else { >+ print "skipping authorities\n" if ( $verbose_logging ); >+ } >+ >+ if ($biblios) { >+ index_records('biblio', $directory, $skip_export, $skip_index, $process_zebraqueue, $as_xml, $noxml, $nosanitize, $do_not_clear_zebraqueue, $verbose_logging, $zebraidx_log_opt, $biblioserverdir); >+ } else { >+ print "skipping biblios\n" if ( $verbose_logging ); >+ } >+} >+ >+# Check the zebra update queue and return true if there are records to process >+sub zebraqueue_not_empty { >+ my $query = $dbh->prepare('SELECT COUNT(*) FROM zebraqueue WHERE done = 0;'); >+ $query->execute; >+ my $count = $query->fetchrow_arrayref->[0]; >+ print "queued records: $count\n" if $verbose_logging > 0; >+ return $count > 0; >+} >+ > # This checks to see if the zebra directories exist under the provided path. > # If they don't, then zebra is likely to spit the dummy. This returns true > # if the directories had to be created, false otherwise. >@@ -692,6 +716,18 @@ Parameters: > > -a index authority records > >+ -daemon Run in daemon mode. The program will loop checking >+ for entries on the zebraqueue table, processing >+ them incrementally if present, and then sleep >+ for a few seconds before repeating the process >+ Checking the zebraqueue table is done with a cheap >+ SQL query. This allows for near realtime update of >+ the zebra search index with low system overhead. >+ Use -sleep to control the checking interval. >+ >+ -sleep 10 Seconcds to sleep between checks of the zebraqueue >+ table in daemon mode. The default is 5 seconds. >+ > -z select only updated and deleted > records marked in the zebraqueue > table. Cannot be used with -r >-- >1.7.9.5 >
You cannot view the attachment while viewing its details because your browser does not support IFRAMEs.
View the attachment on a separate page
.
View Attachment As Diff
View Attachment As Raw
Actions:
View
|
Diff
|
Splinter Review
Attachments on
bug 6435
:
19267
|
19710
|
20574
|
20575
|
21097
|
21099
|
21390
|
21391
|
22039
|
22040
|
22080
|
22201
|
22457
|
22851
|
22859
|
22986