Bug 18098

Summary: Add an index with the count of not onloan items
Product: Koha Reporter: Tomás Cohen Arazi <tomascohen>
Component: SearchingAssignee: Tomás Cohen Arazi <tomascohen>
Status: CLOSED FIXED QA Contact: Marcel de Rooy <m.de.rooy>
Severity: enhancement    
Priority: P5 - low CC: dcook, fridolin.somers, katrin.fischer, kyle, m.de.rooy, nick, tomascohen
Version: master   
Hardware: All   
OS: All   
Change sponsored?: Sponsored Patch complexity: Small patch
Text to go in the release notes:
Version(s) released in:
Bug Depends on: 18208    
Bug Blocks: 19122    
Attachments: Bug 18098: Add an index with the count of available items
Bug 18098: Add an index with the count of not onloan items
[SIGNED-OFF] Bug 18098: Add an index with the count of not onloan items
Bug 18098: Add an index with the count of not onloan items

Description Tomás Cohen Arazi 2017-02-10 17:55:20 UTC

    
Comment 1 Tomás Cohen Arazi 2017-02-10 17:55:56 UTC
There should be an index including the amount of items that are available.
Comment 2 Tomás Cohen Arazi 2017-02-10 18:06:41 UTC
Created attachment 60123 [details] [review]
Bug 18098: Add an index with the count of available items

This patch adds a way to specify which MARC field contains the onloan
status and uses it to generate an XSLT that generates a new index for
Zebra called 'available'. This index contains the amount of items that
have an onloan status different than 1.

The idea is to change the way we build the query on the Perl side, and
just search for available > 0, taking advantage of DOM indexing
capabilities.

To test:
- Have a marcxml record containing holdings (952/995 depending on your
        MARC flavour).
- Run:
  $ xsltproc \
  etc/zebradb/marc_defs/marc21/biblios/biblio-zebra-indexdefs.xsl \
  your_record.marcxml
=> FAIL: There's no 'available' z:index.
- Apply the patch
- Run:
  $ xsltproc \
      etc/zebradb/marc_defs/marc21/biblios/biblio-zebra-indexdefs.xsl \
      your_record.marcxml
=> SUCCESS: There's a <z:index> entry for 'available'
- Sign off :-D

Sponsored-by: ByWater Solutions
Comment 3 Tomás Cohen Arazi 2017-03-03 21:54:40 UTC
Created attachment 60818 [details] [review]
Bug 18098: Add an index with the count of not onloan items

This patch adds a numeric index 'not-onloan-count' containing the value
of 999$x. This subfield is filled by 'rebuild_zebra.pl' by making use of
(bug's 18208) 'EmbedItemsAvailability' filter.

bib1.att and indexes definitions are updated accordingly.

To test:
- Apply the patch
- Pick the right biblio-zebra-indexdefs.xsl file for your setup and
  replace the one your Zebra uses [1]
- Replace your bib1.att
- Replace your ccl.properties
- Have at least one record with more than one item, checkout some
  item(s) from that record(s).
- Rebuild zebra's indexes:
  $ sudo koha-shell kohadev
 k$ cd kohaclone
 k$ misc/migration_tools/rebuild_zebra.pl -r -b -v -k
 (notice the dump directory is kept, you can try the XSLT yourself
  running:
    $ xsltproc \
       etc/zebradb/marc_defs/marc21/biblios/biblio-zebra-indexdefs.xsl \
       /tmp/the_dump_dir/biblios/exported_records | less
=> SUCCESS: There are records with the not-onloan-count index, and the
            value is correct!
- Check Zebra yourself:
  $ yaz-client unix:/var/run/koha/kohadev/bibliosocket
 Z> base biblios
 Z> find @attr 1=9013 @attr 2=5 @attr 4=109 0
=> SUCCESS: The search matches the amount of records with not-onloan
            items.
 Z> s 1+1
=> SUCCESS: Records with 999$x having a value higher than 0 are rendered
- Sign off :-D

Note: While this work is complete on its purpose, it is part of an
attempt to create a better way of filtering by availability.

Sponsored-by: ByWater Solutions

 [1] In kohadevbox this would be
/etc/koha/zebradb/marc_defs/marc21/biblios/biblio-zebra-indexdefs.xsl
Comment 4 Josef Moravec 2017-03-07 10:13:40 UTC
Created attachment 60859 [details] [review]
[SIGNED-OFF] Bug 18098: Add an index with the count of not onloan items

This patch adds a numeric index 'not-onloan-count' containing the value
of 999$x. This subfield is filled by 'rebuild_zebra.pl' by making use of
(bug's 18208) 'EmbedItemsAvailability' filter.

bib1.att and indexes definitions are updated accordingly.

To test:
- Apply the patch
- Pick the right biblio-zebra-indexdefs.xsl file for your setup and
  replace the one your Zebra uses [1]
- Replace your bib1.att
- Replace your ccl.properties
- Have at least one record with more than one item, checkout some
  item(s) from that record(s).
- Rebuild zebra's indexes:
  $ sudo koha-shell kohadev
 k$ cd kohaclone
 k$ misc/migration_tools/rebuild_zebra.pl -r -b -v -k
 (notice the dump directory is kept, you can try the XSLT yourself
  running:
    $ xsltproc \
       etc/zebradb/marc_defs/marc21/biblios/biblio-zebra-indexdefs.xsl \
       /tmp/the_dump_dir/biblios/exported_records | less
=> SUCCESS: There are records with the not-onloan-count index, and the
            value is correct!
- Check Zebra yourself:
  $ yaz-client unix:/var/run/koha/kohadev/bibliosocket
 Z> base biblios
 Z> find @attr 1=9013 @attr 2=5 @attr 4=109 0
=> SUCCESS: The search matches the amount of records with not-onloan
            items.
 Z> s 1+1
=> SUCCESS: Records with 999$x having a value higher than 0 are rendered
- Sign off :-D

Note: While this work is complete on its purpose, it is part of an
attempt to create a better way of filtering by availability.

Sponsored-by: ByWater Solutions

 [1] In kohadevbox this would be
/etc/koha/zebradb/marc_defs/marc21/biblios/biblio-zebra-indexdefs.xsl

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Comment 5 Marcel de Rooy 2017-05-03 14:24:43 UTC
QA: Looking here now
Comment 6 Marcel de Rooy 2017-05-03 14:57:54 UTC
Seems to work fine (MARC21), but we still need biblio-zebra-indexdefs.xsl for unimarc and normarc.
Note that 'working fine' in my case included restarting zebra and indexer.
Comment 7 Tomás Cohen Arazi 2017-05-04 14:27:04 UTC
Created attachment 63112 [details] [review]
Bug 18098: Add an index with the count of not onloan items

This patch adds a numeric index 'not-onloan-count' containing the value
of 999$x. This subfield is filled by 'rebuild_zebra.pl' by making use of
(bug's 18208) 'EmbedItemsAvailability' filter.

bib1.att and indexes definitions are updated accordingly.

To test:
- Apply the patch
- Pick the right biblio-zebra-indexdefs.xsl file for your setup and
  replace the one your Zebra uses [1]
- Replace your bib1.att
- Replace your ccl.properties
- Have at least one record with more than one item, checkout some
  item(s) from that record(s).
- Rebuild zebra's indexes:
  $ sudo koha-shell kohadev
 k$ cd kohaclone
 k$ misc/migration_tools/rebuild_zebra.pl -r -b -v -k
 (notice the dump directory is kept, you can try the XSLT yourself
  running:
    $ xsltproc \
       etc/zebradb/marc_defs/marc21/biblios/biblio-zebra-indexdefs.xsl \
       /tmp/the_dump_dir/biblios/exported_records | less
=> SUCCESS: There are records with the not-onloan-count index, and the
            value is correct!
- Check Zebra yourself:
  $ yaz-client unix:/var/run/koha/kohadev/bibliosocket
 Z> base biblios
 Z> find @attr 1=9013 @attr 2=5 @attr 4=109 0
=> SUCCESS: The search matches the amount of records with not-onloan
            items.
 Z> s 1+1
=> SUCCESS: Records with 999$x having a value higher than 0 are rendered
- Sign off :-D

Note: While this work is complete on its purpose, it is part of an
attempt to create a better way of filtering by availability.

Sponsored-by: ByWater Solutions

 [1] In kohadevbox this would be
/etc/koha/zebradb/marc_defs/marc21/biblios/biblio-zebra-indexdefs.xsl

Edit: Added the missing XSLT changes for UNIMARC and NORMARC

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Comment 8 Kyle M Hall 2017-05-08 13:21:55 UTC
Pushed to master for 17.05, thanks Tomas!
Comment 9 Katrin Fischer 2017-05-14 09:23:37 UTC
This won't get ported back to 16.11.x as it is an enhancement.