Bug 18098 - Add an index with the count of not onloan items
Summary: Add an index with the count of not onloan items
Status: RESOLVED FIXED
Alias: None
Product: Koha
Classification: Unclassified
Component: Searching (show other bugs)
Version: master
Hardware: All All
: P5 - low enhancement (vote)
Assignee: Tomás Cohen Arazi
QA Contact: Marcel de Rooy
URL:
Whiteboard:
Keywords:
Depends on: 18208
Blocks:
  Show dependency treegraph
 
Reported: 2017-02-10 17:55 UTC by Tomás Cohen Arazi
Modified: 2017-05-14 09:23 UTC (History)
7 users (show)

See Also:
Change sponsored?: Sponsored
Patch complexity: Small patch
Bot Control: ---
When did the bot last check this:
Who signed the patch off:
Text to go in the release notes:


Attachments
Bug 18098: Add an index with the count of available items (10.09 KB, patch)
2017-02-10 18:06 UTC, Tomás Cohen Arazi
Details | Diff | Splinter Review
Bug 18098: Add an index with the count of not onloan items (7.22 KB, patch)
2017-03-03 21:54 UTC, Tomás Cohen Arazi
Details | Diff | Splinter Review
[SIGNED-OFF] Bug 18098: Add an index with the count of not onloan items (7.24 KB, patch)
2017-03-07 10:13 UTC, Josef Moravec
Details | Diff | Splinter Review
Bug 18098: Add an index with the count of not onloan items (9.35 KB, patch)
2017-05-04 14:27 UTC, Tomás Cohen Arazi
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Tomás Cohen Arazi 2017-02-10 17:55:20 UTC

    
Comment 1 Tomás Cohen Arazi 2017-02-10 17:55:56 UTC
There should be an index including the amount of items that are available.
Comment 2 Tomás Cohen Arazi 2017-02-10 18:06:41 UTC
Created attachment 60123 [details] [review]
Bug 18098: Add an index with the count of available items

This patch adds a way to specify which MARC field contains the onloan
status and uses it to generate an XSLT that generates a new index for
Zebra called 'available'. This index contains the amount of items that
have an onloan status different than 1.

The idea is to change the way we build the query on the Perl side, and
just search for available > 0, taking advantage of DOM indexing
capabilities.

To test:
- Have a marcxml record containing holdings (952/995 depending on your
        MARC flavour).
- Run:
  $ xsltproc \
  etc/zebradb/marc_defs/marc21/biblios/biblio-zebra-indexdefs.xsl \
  your_record.marcxml
=> FAIL: There's no 'available' z:index.
- Apply the patch
- Run:
  $ xsltproc \
      etc/zebradb/marc_defs/marc21/biblios/biblio-zebra-indexdefs.xsl \
      your_record.marcxml
=> SUCCESS: There's a <z:index> entry for 'available'
- Sign off :-D

Sponsored-by: ByWater Solutions
Comment 3 Tomás Cohen Arazi 2017-03-03 21:54:40 UTC
Created attachment 60818 [details] [review]
Bug 18098: Add an index with the count of not onloan items

This patch adds a numeric index 'not-onloan-count' containing the value
of 999$x. This subfield is filled by 'rebuild_zebra.pl' by making use of
(bug's 18208) 'EmbedItemsAvailability' filter.

bib1.att and indexes definitions are updated accordingly.

To test:
- Apply the patch
- Pick the right biblio-zebra-indexdefs.xsl file for your setup and
  replace the one your Zebra uses [1]
- Replace your bib1.att
- Replace your ccl.properties
- Have at least one record with more than one item, checkout some
  item(s) from that record(s).
- Rebuild zebra's indexes:
  $ sudo koha-shell kohadev
 k$ cd kohaclone
 k$ misc/migration_tools/rebuild_zebra.pl -r -b -v -k
 (notice the dump directory is kept, you can try the XSLT yourself
  running:
    $ xsltproc \
       etc/zebradb/marc_defs/marc21/biblios/biblio-zebra-indexdefs.xsl \
       /tmp/the_dump_dir/biblios/exported_records | less
=> SUCCESS: There are records with the not-onloan-count index, and the
            value is correct!
- Check Zebra yourself:
  $ yaz-client unix:/var/run/koha/kohadev/bibliosocket
 Z> base biblios
 Z> find @attr 1=9013 @attr 2=5 @attr 4=109 0
=> SUCCESS: The search matches the amount of records with not-onloan
            items.
 Z> s 1+1
=> SUCCESS: Records with 999$x having a value higher than 0 are rendered
- Sign off :-D

Note: While this work is complete on its purpose, it is part of an
attempt to create a better way of filtering by availability.

Sponsored-by: ByWater Solutions

 [1] In kohadevbox this would be
/etc/koha/zebradb/marc_defs/marc21/biblios/biblio-zebra-indexdefs.xsl
Comment 4 Josef Moravec 2017-03-07 10:13:40 UTC
Created attachment 60859 [details] [review]
[SIGNED-OFF] Bug 18098: Add an index with the count of not onloan items

This patch adds a numeric index 'not-onloan-count' containing the value
of 999$x. This subfield is filled by 'rebuild_zebra.pl' by making use of
(bug's 18208) 'EmbedItemsAvailability' filter.

bib1.att and indexes definitions are updated accordingly.

To test:
- Apply the patch
- Pick the right biblio-zebra-indexdefs.xsl file for your setup and
  replace the one your Zebra uses [1]
- Replace your bib1.att
- Replace your ccl.properties
- Have at least one record with more than one item, checkout some
  item(s) from that record(s).
- Rebuild zebra's indexes:
  $ sudo koha-shell kohadev
 k$ cd kohaclone
 k$ misc/migration_tools/rebuild_zebra.pl -r -b -v -k
 (notice the dump directory is kept, you can try the XSLT yourself
  running:
    $ xsltproc \
       etc/zebradb/marc_defs/marc21/biblios/biblio-zebra-indexdefs.xsl \
       /tmp/the_dump_dir/biblios/exported_records | less
=> SUCCESS: There are records with the not-onloan-count index, and the
            value is correct!
- Check Zebra yourself:
  $ yaz-client unix:/var/run/koha/kohadev/bibliosocket
 Z> base biblios
 Z> find @attr 1=9013 @attr 2=5 @attr 4=109 0
=> SUCCESS: The search matches the amount of records with not-onloan
            items.
 Z> s 1+1
=> SUCCESS: Records with 999$x having a value higher than 0 are rendered
- Sign off :-D

Note: While this work is complete on its purpose, it is part of an
attempt to create a better way of filtering by availability.

Sponsored-by: ByWater Solutions

 [1] In kohadevbox this would be
/etc/koha/zebradb/marc_defs/marc21/biblios/biblio-zebra-indexdefs.xsl

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Comment 5 Marcel de Rooy 2017-05-03 14:24:43 UTC
QA: Looking here now
Comment 6 Marcel de Rooy 2017-05-03 14:57:54 UTC
Seems to work fine (MARC21), but we still need biblio-zebra-indexdefs.xsl for unimarc and normarc.
Note that 'working fine' in my case included restarting zebra and indexer.
Comment 7 Tomás Cohen Arazi 2017-05-04 14:27:04 UTC
Created attachment 63112 [details] [review]
Bug 18098: Add an index with the count of not onloan items

This patch adds a numeric index 'not-onloan-count' containing the value
of 999$x. This subfield is filled by 'rebuild_zebra.pl' by making use of
(bug's 18208) 'EmbedItemsAvailability' filter.

bib1.att and indexes definitions are updated accordingly.

To test:
- Apply the patch
- Pick the right biblio-zebra-indexdefs.xsl file for your setup and
  replace the one your Zebra uses [1]
- Replace your bib1.att
- Replace your ccl.properties
- Have at least one record with more than one item, checkout some
  item(s) from that record(s).
- Rebuild zebra's indexes:
  $ sudo koha-shell kohadev
 k$ cd kohaclone
 k$ misc/migration_tools/rebuild_zebra.pl -r -b -v -k
 (notice the dump directory is kept, you can try the XSLT yourself
  running:
    $ xsltproc \
       etc/zebradb/marc_defs/marc21/biblios/biblio-zebra-indexdefs.xsl \
       /tmp/the_dump_dir/biblios/exported_records | less
=> SUCCESS: There are records with the not-onloan-count index, and the
            value is correct!
- Check Zebra yourself:
  $ yaz-client unix:/var/run/koha/kohadev/bibliosocket
 Z> base biblios
 Z> find @attr 1=9013 @attr 2=5 @attr 4=109 0
=> SUCCESS: The search matches the amount of records with not-onloan
            items.
 Z> s 1+1
=> SUCCESS: Records with 999$x having a value higher than 0 are rendered
- Sign off :-D

Note: While this work is complete on its purpose, it is part of an
attempt to create a better way of filtering by availability.

Sponsored-by: ByWater Solutions

 [1] In kohadevbox this would be
/etc/koha/zebradb/marc_defs/marc21/biblios/biblio-zebra-indexdefs.xsl

Edit: Added the missing XSLT changes for UNIMARC and NORMARC

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Comment 8 Kyle M Hall 2017-05-08 13:21:55 UTC
Pushed to master for 17.05, thanks Tomas!
Comment 9 Katrin Fischer 2017-05-14 09:23:37 UTC
This won't get ported back to 16.11.x as it is an enhancement.