Attachment 20288 Details for Bug 8507 – Bug 8507: koha-create should be updated to use DOM indexing for bib

[patch] Bug 8507: koha-create should be updated to use DOM indexing for bib

Bug-8507-koha-create-should-be-updated-to-use-DOM-.patch (text/plain), 17.55 KB, created by Chris Cormack on 2013-08-13 09:05:51 UTC

(hide)

Description:

Filename:

MIME Type:

Creator: Chris Cormack

Created: 2013-08-13 09:05:51 UTC

Size: 17.55 KB

patch

obsolete

>From 46e8d2ea1aaa326de70853b5256eca9dcf0e00b1 Mon Sep 17 00:00:00 2001
>From: Tomas Cohen Arazi <tomascohen@gmail.com>
>Date: Thu, 8 Aug 2013 15:15:53 -0300
>Subject: [PATCH] Bug 8507: koha-create should be updated to use DOM indexing
> for bib
>
>This patch makes the koha-create script install the file zebra-biblios-dom.cfg
>with the proper string substitutions inside on the new instance koha-conf.xml file.
>
>It also adds two option switches that control the indexing mode for the instance:
>
> --biblio-idx {dom|grs1}
> --auth-idx {dom|grs1}
>
>DOM indexing is set as the default for both authorities and bibliographic records.
>
>Following drojf (thanks!) advise I arranged stuff like explained here:
>
>  http://wiki.koha-community.org/wiki/Switching_to_dom_indexing
>
>To test:
>- Apply the patch
>- Build your own packages and install them on a test server
>a) Create a new instance without using the new switches like:
> $ koha-create --create-db domtest
> - Check there's a file /etc/koha/sites/domtest/zebra-biblios-dom.cfg
> - Check that /etc/koha/sites/domtest/koha-conf.xml points to:
>   * zebra-biblios-dom.cfg (biblioserver section)
>   * zebra-biblios-dom.cfg (publicserver section)
>   * zebra-authorities-dom.cfg (authorityserver section)
> - Success means the new default is DOM
>b) Play with the 4 possible combination of option switches
> $ koha-create --create-db --auth-idx grs1 --biblio-idx grs1 domtest
> $ koha-create --create-db --auth-idx grs1 --biblio-idx dom domtest
> $ koha-create --create-db --auth-idx dom --biblio-idx grs1 domtest
> $ koha-create --create-db --auth-idx dom --biblio-idx dom domtest
> - Check the koha-conf.xml file reflects the chosen options.
>c) Run
> $ koha-create --help
> - It should advertise this addition accordingly.
>d) Run
> $ man koha-create
> - Man page for koha-create should provide good information on the new switches behaviour
>
>Regards
>To+
>
>Sponsored-by: Universidad Nacional de Cordoba
>Signed-off-by: Mirko Tietgen <mirko@abunchofthings.net>
>Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
>---
> debian/docs/koha-create.xml                    |  16 ++++
> debian/scripts/koha-create                     | 113 ++++++++++++++++++++++++-
> debian/scripts/koha-remove                     |   2 +
> debian/templates/koha-conf-site.xml.in         |  33 ++++----
> debian/templates/zebra-biblios-dom-site.cfg.in |  22 ++---
> 5 files changed, 158 insertions(+), 28 deletions(-)
>
>diff --git a/debian/docs/koha-create.xml b/debian/docs/koha-create.xml
>index 491da62..31c168a 100644
>--- a/debian/docs/koha-create.xml
>+++ b/debian/docs/koha-create.xml
>@@ -27,6 +27,8 @@
>       <arg choice="req"><option>--create-db</option>|<option>--request-db</option>|<option>--populate-db</option>|<option>--use-db</option></arg>
>       <arg><option>--marcflavor</option> marc21|normarc|unimarc</arg>
>       <arg><option>--zebralang</option> en|es|fr|nb|ru|uk</arg>
>+      <arg><option>--auth-idx</option> dom|grs1</arg>
>+      <arg><option>--biblio-idx</option> dom|grs1</arg>
>       <arg><option>--defaultsql</option> /path/to/some.sql</arg>
>       <arg><option>--configfile</option> /path/to/config</arg>
>       <arg><option>--passwdfile</option> /path/to/passwd</arg>
>@@ -109,6 +111,20 @@
>       </listitem>
>     </varlistentry>
> 
>+    <varlistentry>
>+      <term><option>--auth-idx</option></term>
>+      <listitem>
>+        <para>Specified the desired indexing mode for authority records. Valid options are <option>dom</option> (default) and <option>grs1</option>.</para>
>+      </listitem>
>+    </varlistentry>
>+
>+    <varlistentry>
>+      <term><option>--biblio-idx</option></term>
>+      <listitem>
>+        <para>Specified the desired indexing mode for bibliographic records. Valid options are <option>dom</option> (default) and <option>grs1</option>.</para>
>+      </listitem>
>+    </varlistentry>
>+
>   </variablelist>
>   </refsect1>
> 
>diff --git a/debian/scripts/koha-create b/debian/scripts/koha-create
>index a0a830a..e8b2578 100755
>--- a/debian/scripts/koha-create
>+++ b/debian/scripts/koha-create
>@@ -1,4 +1,4 @@
>-#!/bin/sh
>+#!/bin/bash
> #
> # koha-create -- Create a new Koha instance.
> # Copyright 2010  Catalyst IT, Ltd
>@@ -22,6 +22,7 @@ set -e
> usage="Usage: $0 [--create-db|--request-db|--populate-db|--use-db] \
>     [--marcflavor marc21|normarc|unimarc] \
>     [--zebralang en|es|fr|nb|ru|uk] \
>+    [--auth-idx dom|grs1] [--biblio-idx dom|grs1] \
>     [--defaultsql /path/to/some.sql] \
>     [--configfile /path/to/config] [--passwdfile /path/to/passwd] \
>     [--database database] [--adminuser n] instancename"
>@@ -45,6 +46,14 @@ generate_config_file() {
>         -e "s/__ZEBRA_PASS__/$zebrapwd/g" \
>         -e "s/__ZEBRA_MARC_FORMAT__/$ZEBRA_MARC_FORMAT/g" \
>         -e "s/__ZEBRA_LANGUAGE__/$ZEBRA_LANGUAGE/g" \
>+        -e "s/__BIBLIOS_INDEXING_MODE__/$BIBLIOS_INDEXING_MODE/g" \
>+        -e "s/__AUTHORITIES_INDEXING_MODE__/$AUTHORITIES_INDEXING_MODE/g" \
>+        -e "s/__ZEBRA_BIBLIOS_CFG__/$ZEBRA_BIBLIOS_CFG/g" \
>+        -e "s/__ZEBRA_AUTHORITIES_CFG__/$ZEBRA_AUTHORITIES_CFG/g" \
>+        -e "s/__START_BIBLIOS_RETRIEVAL_INFO__/`echo $START_BIBLIOS_RETRIEVAL_INFO`/g" \
>+        -e "s/__END_BIBLIOS_RETRIEVAL_INFO__/`echo $END_BIBLIOS_RETRIEVAL_INFO`/g" \
>+        -e "s/__START_AUTHORITIES_RETRIEVAL_INFO__/`echo $START_AUTHORITIES_RETRIEVAL_INFO`/g" \
>+        -e "s/__END_AUTHORITIES_RETRIEVAL_INFO__/`echo $END_AUTHORITIES_RETRIEVAL_INFO`/g" \
>         -e "s/__DB_NAME__/$mysqldb/g" \
>         -e "s/__DB_HOST__/$mysqlhost/g" \
>         -e "s/__DB_USER__/$mysqluser/g" \
>@@ -53,6 +62,7 @@ generate_config_file() {
>         -e "s/__UNIXGROUP__/$username/g" \
>         -e "s/__PLUGINS_DIR__/\/var\/lib\/koha\/$name\/plugins/g" \
>         "/etc/koha/$1" > "$2"
>+
> }
> 
> getmysqlhost() {
>@@ -75,6 +85,75 @@ getinstancemysqldatabase() {
>     xmlstarlet sel -t -v 'yazgfs/config/database' "/etc/koha/sites/$1/koha-conf.xml"
> }
> 
>+set_biblios_indexing_mode()
>+{
>+    local indexing_mode=$1
>+    local marc_format=$2
>+
>+    case $indexing_mode in
>+        "dom")
>+            START_BIBLIOS_RETRIEVAL_INFO=`cat <<EOF
>+    <xi:include href="\/etc\/koha\/$marc_format-retrieval-info-bib-dom.xml"\n
>+                xmlns:xi="http:\/\/www.w3.org\/2001\/XInclude">\n
>+     <xi:fallback>\n
>+      <retrievalinfo>
>+EOF`
>+
>+            END_BIBLIOS_RETRIEVAL_INFO=`cat <<EOF
>+      <\/retrievalinfo>\n
>+     <\/xi:fallback>\n
>+    <\/xi:include>
>+EOF`
>+            BIBLIOS_INDEXING_MODE="dom"
>+            ZEBRA_BIBLIOS_CFG="zebra-biblios-dom.cfg"
>+            ;;
>+        "grs1")
>+            START_BIBLIOS_RETRIEVAL_INFO="    <retrievalinfo>"
>+            END_BIBLIOS_RETRIEVAL_INFO="    <\/retrievalinfo>"
>+            BIBLIOS_INDEXING_MODE="grs1"
>+            ZEBRA_BIBLIOS_CFG="zebra-biblios.cfg"
>+            ;;
>+        *)
>+            die "Error: '$indexing_mode' is not a valid indexing mode for bibliographic records."
>+            ;;
>+    esac
>+}
>+
>+
>+set_authorities_indexing_mode()
>+{
>+    local indexing_mode=$1
>+    local marc_format=$2
>+
>+    case $indexing_mode in
>+        "dom")
>+            START_AUTHORITIES_RETRIEVAL_INFO=`cat <<EOF
>+    <xi:include href="\/etc\/koha\/$marc_format-retrieval-info-auth-dom.xml"\n
>+                xmlns:xi="http:\/\/www.w3.org\/2001\/XInclude">\n
>+     <xi:fallback>\n
>+      <retrievalinfo>
>+EOF`
>+
>+            END_AUTHORITIES_RETRIEVAL_INFO=`cat <<EOF
>+      <\/retrievalinfo>\n
>+     <\/xi:fallback>\n
>+    <\/xi:include>\n
>+EOF`
>+            AUTHORITIES_INDEXING_MODE="dom"
>+            ZEBRA_AUTHORITIES_CFG="zebra-authorities-dom.cfg"
>+            ;;
>+        "grs1")
>+            START_AUTHORITIES_RETRIEVAL_INFO="    <retrievalinfo>"
>+            END_AUTHORITIES_RETRIEVAL_INFO="    <\/retrievalinfo>"
>+            AUTHORITIES_INDEXING_MODE="grs1"
>+            ZEBRA_AUTHORITIES_CFG="zebra-authorities.cfg"
>+            ;;
>+        *)
>+            die "Error: '$indexing_mode' is not a valid indexing mode for authority records."
>+            ;;
>+    esac
>+}
>+
> # Set defaults and read config file, if it exists.
> DOMAIN=""
> OPACPORT="80"
>@@ -88,6 +167,16 @@ ZEBRA_MARC_FORMAT="marc21"
> ZEBRA_LANGUAGE="en"
> ADMINUSER="1"
> PASSWDFILE="/etc/koha/passwd"
>+
>+# Indexing mode variables (default is DOM)
>+BIBLIOS_INDEXING_MODE="dom"
>+AUTHORITIES_INDEXING_MODE="dom"
>+
>+START_BIBLIOS_RETRIEVAL_INFO=""
>+END_BIBLIOS_RETRIEVAL_INFO=""
>+START_AUTHORITIES_RETRIEVAL_INFO=""
>+END_AUTHORITIES_RETRIEVAL_INFO=""
>+
> if [ -e /etc/koha/koha-sites.conf ]
> then
>     . /etc/koha/koha-sites.conf
>@@ -95,7 +184,7 @@ fi
> 
> [ $# -ge 2 ] && [ $# -le 16 ] || die $usage
> 
>-TEMP=`getopt -o crpm:l:d:f:b:a: -l create-db,request-db,populate-db,use-db,marcflavor:,zebralang:,defaultsql:,configfile:,passwdfile:,database:,adminuser: \
>+TEMP=`getopt -o crpm:l:d:f:b:a: -l create-db,request-db,populate-db,use-db,marcflavor:,auth-idx:,biblio-idx:,zebralang:,defaultsql:,configfile:,passwdfile:,database:,adminuser: \
>      -n "$0" -- "$@"`
> 
> # Note the quotes around `$TEMP': they are essential!
>@@ -106,6 +195,9 @@ CLO_ZEBRA_MARC_FORMAT=""
> CLO_ZEBRA_LANGUAGE=""
> CLO_DEFAULTSQL=""
> CLO_ADMINUSER=""
>+CLO_BIBLIOS_INDEXING_MODE=""
>+CLO_AUTHORITIES_INDEXING_MODE=""
>+
> 
> while true ; do
> 	case "$1" in
>@@ -115,6 +207,8 @@ while true ; do
>         -u|--use-db) op=use ; shift ;;
> 		-m|--marcflavor) CLO_ZEBRA_MARC_FORMAT="$2" ; shift 2 ;;
> 		-l|--zebralang) CLO_ZEBRA_LANGUAGE="$2" ; shift 2 ;;
>+		--auth-idx) CLO_AUTHORITIES_INDEXING_MODE="$2" ; shift 2 ;;
>+		--biblio-idx) CLO_BIBLIOS_INDEXING_MODE="$2" ; shift 2 ;;
> 		-d|--defaultsql) CLO_DEFAULTSQL="$2" ; shift 2 ;;
> 		-f|--configfile) configfile="$2" ; shift 2 ;;
>         -s|--passwdfile) CLO_PASSWDFILE="$2" ; shift 2 ;;
>@@ -158,6 +252,19 @@ then
>     PASSWDFILE="$CLO_PASSWDFILE"
> fi
> 
>+if [ "$CLO_BIBLIOS_INDEXING_MODE" !=  "" ]; then
>+    BIBLIOS_INDEXING_MODE=$CLO_BIBLIOS_INDEXING_MODE
>+fi
>+
>+set_biblios_indexing_mode $BIBLIOS_INDEXING_MODE $ZEBRA_MARC_FORMAT
>+
>+
>+if [ "$CLO_AUTHORITIES_INDEXING_MODE" !=  "" ]; then
>+    AUTHORITIES_INDEXING_MODE=$CLO_AUTHORITIES_INDEXING_MODE
>+fi
>+
>+set_authorities_indexing_mode $AUTHORITIES_INDEXING_MODE $ZEBRA_MARC_FORMAT
>+
> name="$1"
> 
> opacdomain="$OPACPREFIX$name$OPACSUFFIX$DOMAIN"
>@@ -261,6 +368,8 @@ eof
>     # Generate and install Zebra config files.
>     generate_config_file zebra-biblios-site.cfg.in \
>         "/etc/koha/sites/$name/zebra-biblios.cfg"
>+    generate_config_file zebra-biblios-dom-site.cfg.in \
>+        "/etc/koha/sites/$name/zebra-biblios-dom.cfg"
>     generate_config_file zebra-authorities-site.cfg.in \
>         "/etc/koha/sites/$name/zebra-authorities.cfg"
>     generate_config_file zebra-authorities-dom-site.cfg.in \
>diff --git a/debian/scripts/koha-remove b/debian/scripts/koha-remove
>index d68aebc..3fa5ef1 100755
>--- a/debian/scripts/koha-remove
>+++ b/debian/scripts/koha-remove
>@@ -74,6 +74,8 @@ eof
>         rm "/etc/koha/sites/$name/koha-conf.xml"
>     [ -f "/etc/koha/sites/$name/zebra-biblios.cfg" ] && \
>         rm "/etc/koha/sites/$name/zebra-biblios.cfg"
>+    [ -f "/etc/koha/sites/$name/zebra-biblios-dom.cfg" ] && \
>+        rm "/etc/koha/sites/$name/zebra-biblios-dom.cfg"
>     [ -f "/etc/koha/sites/$name/zebra-authorities.cfg" ] && \
>         rm "/etc/koha/sites/$name/zebra-authorities.cfg"
>     [ -f "/etc/koha/sites/$name/zebra-authorities-dom.cfg" ] && \
>diff --git a/debian/templates/koha-conf-site.xml.in b/debian/templates/koha-conf-site.xml.in
>index ab8aff5..80da24a 100644
>--- a/debian/templates/koha-conf-site.xml.in
>+++ b/debian/templates/koha-conf-site.xml.in
>@@ -18,7 +18,7 @@
> <listen id="mergeserver">tcp:@:__MERGE_SERVER_PORT__</listen>
> <server id="mergeserver"  listenref="mergeserver">
>     <directory>/var/lib/koha/__KOHASITE__/biblios</directory>
>-    <config>/var/lib/koha/__KOHASITE__/zebra-biblios.cfg</config>
>+    <config>/var/lib/koha/__KOHASITE__/__ZEBRA_BIBLIOS_CFG__</config>
>     <cql2rpn>/var/lib/koha/__KOHASITE__/pqf.properties</cql2rpn>
> </server>
> -->
>@@ -26,9 +26,10 @@
> 
> <server id="biblioserver"  listenref="biblioserver">
>     <directory>/var/lib/koha/__KOHASITE__/biblios</directory>
>-    <config>/etc/koha/sites/__KOHASITE__/zebra-biblios.cfg</config>
>+    <config>/etc/koha/sites/__KOHASITE__/__ZEBRA_BIBLIOS_CFG__</config>
>     <cql2rpn>/etc/koha/zebradb/pqf.properties</cql2rpn>
>-     <retrievalinfo>
>+
>+    __START_BIBLIOS_RETRIEVAL_INFO__
>        <retrieval syntax="__ZEBRA_MARC_FORMAT__" name="F"/>
>        <retrieval syntax="__ZEBRA_MARC_FORMAT__" name="B"/>
>        <retrieval syntax="xml" name="F"/>
>@@ -75,7 +76,8 @@
>            <xslt stylesheet="/usr/share/koha/intranet/htdocs/intranet-tmpl/prog/en/xslt/MARC21slimUtils.xsl"/>
>          </backend>
>        </retrieval>
>-     </retrievalinfo>
>+    __END_BIBLIOS_RETRIEVAL_INFO__
>+
>     
> <server id="authorityserver"  listenref="authorityserver" >
>     <directory>/var/lib/koha/__KOHASITE__/authorities</directory>
>-    <config>/etc/koha/sites/__KOHASITE__/zebra-authorities-dom.cfg</config>
>+    <config>/etc/koha/sites/__KOHASITE__/__ZEBRA_AUTHORITIES_CFG__</config>
>     <cql2rpn>/etc/koha/zebradb/pqf.properties</cql2rpn>
>-    <xi:include href="/etc/koha/__ZEBRA_MARC_FORMAT__-retrieval-info-auth-dom.xml"
>-                xmlns:xi="http://www.w3.org/2001/XInclude">
>-      <xi:fallback>
>-     <retrievalinfo>
>+
>+    __START_AUTHORITIES_RETRIEVAL_INFO__
>        <retrieval syntax="__ZEBRA_MARC_FORMAT__" name="F"/>
>        <retrieval syntax="__ZEBRA_MARC_FORMAT__" name="B"/>
>        <retrieval syntax="xml" name="marcxml"
>@@ -145,9 +145,8 @@
>            <xslt stylesheet="/usr/share/koha/intranet/htdocs/intranet-tmpl/prog/en/xslt/MARC21slimUtils.xsl"/>
>          </backend>
>        </retrieval>
>-     </retrievalinfo>
>-      </xi:fallback>
>-    </xi:include>
>+    __END_AUTHORITIES_RETRIEVAL_INFO__
>+
>     
>  <memcached_servers></memcached_servers>
>  <memcached_namespace></memcached_namespace>
>+ <zebra_bib_index_mode>__BIBLIOS_INDEXING_MODE__</zebra_bib_index_mode>
>+ <zebra_auth_index_mode>__AUTHORITIES_INDEXING_MODE__</zebra_auth_index_mode>
>  <queryparser_config>/etc/koha/searchengine/queryparser.yaml</queryparser_config>
> </config>
> 
>diff --git a/debian/templates/zebra-biblios-dom-site.cfg.in b/debian/templates/zebra-biblios-dom-site.cfg.in
>index a748ddd..88d0d6d 100644
>--- a/debian/templates/zebra-biblios-dom-site.cfg.in
>+++ b/debian/templates/zebra-biblios-dom-site.cfg.in
>@@ -3,7 +3,7 @@
> # $Id: zebra.cfg,v 1.1.2.2 2006/05/09 12:03:16 rangi Exp $
> #
> # Where are the config files located?
>-profilePath:__ZEBRA_CONF_DIR__/biblios/etc:__ZEBRA_CONF_DIR__/etc:__ZEBRA_CONF_DIR__/marc_defs/__ZEBRA_MARC_FORMAT__/biblios:__ZEBRA_CONF_DIR__/lang_defs/__ZEBRA_LANGUAGE__:__ZEBRA_CONF_DIR__/xsl
>+profilePath:/etc/koha/zebradb/biblios/etc:/etc/koha/zebradb/etc:/etc/koha/zebradb/marc_defs/__ZEBRA_MARC_FORMAT__/biblios:/etc/koha/zebradb/lang_defs/__ZEBRA_LANGUAGE__:/etc/koha/zebradb/xsl
> # modulePath - where to look for loadable zebra modules
> modulePath: /usr/lib/idzebra-2.0/modules
> 
>@@ -21,9 +21,9 @@ attset: gils.att
> # http://www.indexdata.dk/zebra/doc/zebra-cfg.tkl
> # http://www.indexdata.dk/zebra/doc/grs.tkl
> 
>-recordtype: dom.__ZEBRA_CONF_DIR__/biblios/etc/dom-config.xml
>-marcxml.recordtype: dom.__ZEBRA_CONF_DIR__/biblios/etc/dom-config.xml
>-iso2709.recordtype: dom.__ZEBRA_CONF_DIR__/biblios/etc/dom-config-marc.xml
>+recordtype: dom./etc/koha/zebradb/biblios/etc/dom-config.xml
>+marcxml.recordtype: dom./etc/koha/zebradb/biblios/etc/dom-config.xml
>+iso2709.recordtype: dom./etc/koha/zebradb/biblios/etc/dom-config-marc.xml
> 
> recordId: (bib1,Local-number)
> storeKeys:1
>@@ -31,18 +31,18 @@ storeData:1
> 
> 
> # Lock File Area
>-lockDir: __ZEBRA_LOCK_DIR__/biblios
>+lockDir: /var/lock/koha/__KOHASITE__/biblios
> perm.anonymous:ar
>-perm.__ZEBRA_USER__:rw
>-passwd: __ZEBRA_CONF_DIR__/etc/passwd
>-register: __ZEBRA_DATA_DIR__/biblios/register:20G
>-shadow: __ZEBRA_DATA_DIR__/biblios/shadow:20G
>+perm.kohauser:rw
>+passwd: /etc/koha/sites/__KOHASITE__/zebra.passwd
>+register: /var/lib/koha/__KOHASITE__/biblios/register:20G
>+shadow: /var/lib/koha/__KOHASITE__/biblios/shadow:20G
> 
> # Temp File area for result sets
>-setTmpDir: __ZEBRA_DATA_DIR__/biblios/tmp
>+setTmpDir: /var/lib/koha/__KOHASITE__/biblios/tmp
> 
> # Temp File area for index program
>-keyTmpDir: __ZEBRA_DATA_DIR__/biblios/key
>+keyTmpDir: /var/lib/koha/__KOHASITE__/biblios/key
> 
> # Approx. Memory usage during indexing
> memMax: 50M
>-- 
>1.8.4.rc1

Actions: View | Diff | Splinter Review

Attachments on bug 8507: 20182 | 20189 | 20212 | 20247 | 20251 | 20256 | 20263 | 20288 | 20289