Bug 11677

Summary: Limit to Only items currently available for loan or reference not working
Product: Koha Reporter: Kyle M Hall <kyle>
Component: SearchingAssignee: Nick Clemens <nick>
Status: CLOSED FIXED QA Contact: Tomás Cohen Arazi <tomascohen>
Severity: major    
Priority: P5 - low CC: barton, cbrannon, clackman, crohde, edischer, eveal, fridolin.somers, gaetan.boisson, george, hayleypelham, jean-manuel.broust, katrin.fischer, lucas, magnus, margaret, marjorie.barry-vila, martin.renvoize, mfuerst, nick, olli-antti.kivilahti, p.thrasher, patrick.robitaille, rcoert, rkuiper, sally.healey, thysia, tomascohen
Version: master   
Hardware: All   
OS: All   
See Also: http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=5958
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20408
Change sponsored?: --- Patch complexity: Trivial patch
Documentation contact: Documentation submission:
Text to go in the release notes:
Version(s) released in:
19.11.00,19.05.04
Bug Depends on:    
Bug Blocks: 20396    
Attachments: Bug 11677 - Limit to Only items currently available for loan or reference not working!
Bug 11677 - (KD-157) Limit to Only items currently available for loan or reference not working!
Bug 11677 - Limit to Only items currently available for loan or reference not working!
Bug 11677 - Limit to Only items currently available for loan or reference not working!
Bug 11677 - Limit to Only items currently available for loan or reference not working!
Bug 11677: Use zebra not-onloan-count index for availability limit
[ALTERNATE] Bug 11677: Use zebra not-onloan-count index for availability limit
Bug 11677: Use zebra not-onloan-count index for availability limit
Bug 11677: Use zebra not-onloan-count index for availability limit
Bug 11677: Use zebra not-onloan-count index for availability limit

Description Kyle M Hall 2014-02-04 12:29:52 UTC
The ability to limit a search to only available items does not seem to be working as expected. Sometimes records with available items are not included in the results.

Example: http://screencast.com/t/OVgYybzLE
Comment 1 Peggy Thrasher 2014-02-04 14:19:19 UTC
I can cause this problem as follows:

Search in the OPAC
Click on a title to take you to the detail page
Click on the author's name or a subject heading
When you have the new summary screen with the author's or subject's listings, click on limit to Currently Available.

Then bad things happen. Looks like it is searching for "available" in one instance, doesn't find anything in other instances, or displays the wrong list (1 item instead of 291 results)
Comment 3 Barton Chittenden 2014-07-02 10:39:21 UTC
I have replicated this issue at a partner's library, in this case the full search showed both books and audio disks available. When the 'limit to available' facet is selected, only available audio books are returned by the search.
Comment 4 Barton Chittenden 2014-07-02 10:46:06 UTC
Examples from OPAC:

Search for "Alex+Rider+Horowitz" shows records of type "book" and "sound", both available and un-available.

http://catalog.losgatosca.gov/cgi-bin/koha/opac-search.pl?q=Alex+Rider+Horowitz

Clicking "Limit to currently available items."

Shows only available items of type "sound".

http://catalog.losgatosca.gov/cgi-bin/koha/opac-search.pl?idx=kw&q=Alex%20Rider%20Horowitz&sort_by=relevance_dsc&limit=available
Comment 5 Olli-Antti Kivilahti 2014-07-18 08:29:39 UTC
We have the same issue when making a keyword search, then faceting by branch, then limiting by availability.
Limiting by availability currently seems to display only results for which all items are available, even if one item is checked out, the whole result is filtered.

What is needed is a feature to limit by availability for a certain branch, not just general availability.
Comment 6 Olli-Antti Kivilahti 2014-07-18 17:39:35 UTC Comment hidden (obsolete)
Comment 7 Olli-Antti Kivilahti 2014-07-18 17:40:12 UTC
Sharing is caring!
Comment 8 Olli-Antti Kivilahti 2014-10-09 15:44:06 UTC Comment hidden (obsolete)
Comment 9 Olli-Antti Kivilahti 2014-10-09 15:53:39 UTC Comment hidden (obsolete)
Comment 10 Olli-Antti Kivilahti 2014-10-21 10:50:41 UTC Comment hidden (obsolete)
Comment 11 Tomás Cohen Arazi 2014-11-11 18:18:18 UTC
As we have deprecated GRS-1, we could just focus on DOM, and implement this using XSLT's for calculating, instead of crafting the search string.
Comment 12 Olli-Antti Kivilahti 2014-12-15 15:31:35 UTC
Created attachment 34431 [details] [review]
Bug 11677 - Limit to Only items currently available for loan or reference not working!

This row:
$availability_limit .=
"( ( allrecords,AlwaysMatches='' not onloan,AlwaysMatches='') and (lost,st-numeric=0) )";
Causes all records in Zebra to be selected and removes records with even one item onloan
  (even if items would be available)!
This is obviously wrong, but because Zebra (and not many other text indexers) cannot deal
with (If undef)-search terms, a fix might be to set 952$q (onloan) datetime to 0000-00-00
instead of NULL in the DB. Then we could find Records with items where onloan is 0000-00-00.

Comments out that Zebra availability search and deals with availability on Koha-side.

Gets the branch or holdingbranch limit from opac-search.pl or search.pl and uses that to
skip Zebra results which are onloan or notforloan.
Updates the result set size when results are removed.

Fixed a bug where pagination broke, due to the $results_hash->{'RECORDS'} -array getting populated
always starting from 0, where it should have started from the given $offset.
Comment 13 Olli-Antti Kivilahti 2014-12-15 15:33:29 UTC
Hi there! We have this in production now for ~2-3 months.
Works fine and doesn't cause too much of a slowdown.

I recommend XSLT-magic to properly solve this issue. Back then I didn't know XSLT yet to make logic expressions with it but it would definetely be the best alternative.
Comment 14 Gaetan Boisson 2018-03-13 13:44:13 UTC
*** Bug 14972 has been marked as a duplicate of this bug. ***
Comment 15 Gaetan Boisson 2018-03-13 13:44:42 UTC
*** Bug 13715 has been marked as a duplicate of this bug. ***
Comment 16 Gaetan Boisson 2018-03-14 13:46:33 UTC
This is still an issue. I propose to remove this facet as well as the equivalent in the advanced search form until this is fixed.
Comment 17 Gaetan Boisson 2018-03-14 16:31:05 UTC
I have created a new ticket for removing the feature.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20408

This ticket can remain open in order to fix the feature eventually.
Comment 18 Ed Veal 2018-07-30 14:19:38 UTC
This is a long standing issue and should be address or the feature removed. It is very confusing to patrons. 

Ed
Comment 19 Kyle M Hall 2019-02-06 15:22:07 UTC
Tomas added a new availability index that could be used to fix this bug. This will require modifying the CCL query used to check availability.
Comment 20 Christopher Brannon 2019-07-10 16:00:56 UTC
Is this going to get fixed???
Comment 21 Chris Rohde 2019-07-10 16:11:15 UTC
Yes, this is pretty basic OPAC functionality--can someone please take a look at fixing this bug?
Comment 22 Rhonda Kuiper 2019-07-10 17:56:29 UTC
I get comments on this broken function regularly.  Our staff would LOVE to have this fixed.  I have voted for this fix.
Comment 23 George Williams (NEKLS) 2019-07-11 21:36:25 UTC
(In reply to Kyle M Hall from comment #19)
> Tomas added a new availability index that could be used to fix this bug.
> This will require modifying the CCL query used to check availability.

Any chance that the work Tomas has done can be used to fix this long broken feature that is seems to be a working part of every other ILS?
Comment 24 Martha Fuerst 2019-07-17 16:03:44 UTC
I thought I had left a comment on this, but I'm not seeing it.

We really need to get this fixed. Add me to the list of squeaky wheels and volunteers to test for SO. Three years for what is basic functionality for pretty much any current ILS/LMS? It's shameful.

This needs attention, and the sooner the better.
Comment 25 Nick Clemens 2019-08-11 16:14:22 UTC
Created attachment 92144 [details] [review]
Bug 11677: Use zebra not-onloan-count index for availability limit

This uses the work from bug 18098 to use the count of not-onloan items to
determine availability.

Note that availability in this sense means not check out and not lost. Transfers, holds, and not for loan statuses
should probably have indexes built in the same way as bug 18098 (there are related fixmes in the code)

To test:
 1 - make sure SearchEngine pref is set to 'Zebra'
 2 - Do a search on the OPAC that returns some records
 3 - Find a record in those results with more than one 1 item, and checkout 1 leaving 1 (or more) available
 4 - Find another record with only 1 item, check that item out
 5 - Limit opac search to available items - both records disappear
 6 - Apply patch
 7 - Restart all the things
 8 - Repeat search
 9 - First record should show, but no second
10 - Mark other items of first record as lost, return items on second record
11 - Repeat search, first record should not be in results, second should
Comment 26 Hayley Pelham 2019-08-12 05:25:08 UTC
Patch doesn't apply

git bz apply 11677
Bug 11677 - Limit to Only items currently available for loan or reference not working

34431 - Bug 11677 - Limit to Only items currently available for loan or reference not working!
92144 - Bug 11677: Use zebra not-onloan-count index for availability limit

Apply? [(y)es, (n)o, (i)nteractive] y
Applying: Bug 11677 - Limit to Only items currently available for loan or reference not working!
error: sha1 information is lacking or useless (C4/Search.pm).
error: could not build fake ancestor
Patch failed at 0001 Bug 11677 - Limit to Only items currently available for loan or reference not working!
Comment 27 Tomás Cohen Arazi 2019-08-12 13:46:06 UTC
Created attachment 92154 [details] [review]
[ALTERNATE] Bug 11677: Use zebra not-onloan-count index for availability limit

This patch makes the original filter for available items use the
'not-onloan-count' index instead of the 'onloan' one. It keeps the
allrecords,AlwaysMatches='' portion of the query so it still catches
records with the tested indexed missing.
Comment 28 Katrin Fischer 2019-08-13 05:36:56 UTC
Do we have 3 alternate patches here now?
Comment 29 Tomás Cohen Arazi 2019-08-13 12:38:44 UTC
(In reply to Katrin Fischer from comment #28)
> Do we have 3 alternate patches here now?

We have discussed it with Nick, and both our patches return the same results. That's because lost won't be NULL (itemlost is defined as NOT NULL) and so there will always be the lost index on records. The same applies to the not-onloan-count index, which is generated on indexing. Way to test:

kohadev-koha@180acd7da540:/kohadevbox/koha$ yaz-client unix://var/run/koha/kohadev/bibliosocket
Connecting...OK.
Sent initrequest.
Connection accepted by v3 target.
ID     : 81
Name   : Zebra Information Server/GFS/YAZ
Version: 4.2.30 98864b44c654645bc16b2c54f822dc2e45a93031
Options: search present delSet triggerResourceCtrl scan sort extendedServices namedResultSets
Elapsed: 0.004222
Z> base biblios
Z> querytype ccl2rpn
Z> set_cclfile /etc/koha/zebradb/ccl.properties
Z> f ( (not-onloan-count,st-numeric,ge = 1) and (lost,st-numeric=0) )
Sent searchRequest.
Received SearchResponse.
Search was a success.
Number of hits: 411, setno 1
SearchResult-1: term=1 cnt=411, term=0 cnt=411
records returned: 0
Elapsed: 0.003008
Z> f ( (allrecords,AlwaysMatches='') and (not-onloan-count,st-numeric,ge = 1) and (lost,st-numeric=0) )
Sent searchRequest.
Received SearchResponse.
Search was a success.
Number of hits: 411, setno 2
SearchResult-1: term= cnt=436, term=1 cnt=411, term=0 cnt=411
records returned: 0
Elapsed: 0.000761
Z>

He's working on fixing the tests ATM. I will obsolete mine and Olli's.
Comment 30 Tomás Cohen Arazi 2019-08-13 12:41:09 UTC
(In reply to Tomás Cohen Arazi from comment #29)
> (In reply to Katrin Fischer from comment #28)
> > Do we have 3 alternate patches here now?
> 
> We have discussed it with Nick, and both our patches return the same
> results. That's because lost won't be NULL (itemlost is defined as NOT NULL)
> and so there will always be the lost index on records. The same applies to
> the not-onloan-count index, which is generated on indexing. Way to test:

I forgot to mention that the lost index CAN be absent on a record, when it has no items. But that means it has no items available so it is ok :-D
Comment 31 Nick Clemens 2019-08-13 13:42:45 UTC
Created attachment 92176 [details] [review]
Bug 11677: Use zebra not-onloan-count index for availability limit

This uses the work from bug 18098 to use the count of not-onloan items to
determine availability.

It keeps the allrecords,AlwaysMatches='' portion of the query so it still catches
records with the tested indexed missing.
These shouldn't exist except in the case of missing data, but kept for future use as well as
it has no negative impact and minimizes the change made.

Note that availability in this sense means not check out and not lost. Transfers, holds, and not for loan statuses
should probably have indexes built in the same way as bug 18098 (there are related FIXMEs in the code)

This updates the export_records file used for testing - adding three records with a not-onloan-count (999$x) of 1,
but marking the item on the third record lost to prevent its return

To test:
   1 - make sure SearchEngine pref is set to 'Zebra'
   2 - Do a search on the OPAC that returns some records
   3 - Find a record in those results with more than one 1 item, and checkout 1 leaving 1 (or more) available
   4 - Find another record with only 1 item, check that item out
   5 - Limit opac search to available items - both records disappear
   6 - Apply patch
   7 - Restart all the things
   8 - Repeat search
   9 - First record should show, but no second
  10 - Mark other items of first record as lost, return items on second record
  11 - Repeat search, first record should not be in results, second should
  12 - prove -v t/db_dependent/Search.t
Comment 32 Hayley Pelham 2019-08-13 21:33:20 UTC
(In reply to Nick Clemens from comment #31)
> Created attachment 92176 [details] [review] [review]
> Bug 11677: Use zebra not-onloan-count index for availability limit
> 
> This uses the work from bug 18098 to use the count of not-onloan items to
> determine availability.
> 
> It keeps the allrecords,AlwaysMatches='' portion of the query so it still
> catches
> records with the tested indexed missing.
> These shouldn't exist except in the case of missing data, but kept for
> future use as well as
> it has no negative impact and minimizes the change made.
> 
> Note that availability in this sense means not check out and not lost.
> Transfers, holds, and not for loan statuses
> should probably have indexes built in the same way as bug 18098 (there are
> related FIXMEs in the code)
> 
> This updates the export_records file used for testing - adding three records
> with a not-onloan-count (999$x) of 1,
> but marking the item on the third record lost to prevent its return
> 
> To test:
>    1 - make sure SearchEngine pref is set to 'Zebra'
>    2 - Do a search on the OPAC that returns some records
>    3 - Find a record in those results with more than one 1 item, and
> checkout 1 leaving 1 (or more) available
>    4 - Find another record with only 1 item, check that item out
>    5 - Limit opac search to available items - both records disappear
>    6 - Apply patch
>    7 - Restart all the things
>    8 - Repeat search
>    9 - First record should show, but no second
>   10 - Mark other items of first record as lost, return items on second
> record
>   11 - Repeat search, first record should not be in results, second should
>   12 - prove -v t/db_dependent/Search.t

Hi Nick,

All looks to work as expected, but the tests did not pass for me. Here is the output, please let me know if there is something I have missed:

kohadev-koha@kohadevbox:/home/vagrant/kohaclone$ prove -v t/db_dependent/Search.t
t/db_dependent/Search.t .. 
1..2
# Subtest: MARC21 + DOM
    1..112
    ok 1 - use C4::Search;
    ok 2 - Title index supported
    ok 3 - Accelerated reading level index supported
    ok 4 - Accelerated reading point index supported
    ok 5 - Found duplicate with ISBN
    ok 6 - Found duplicate with author/title
    ok 7 - SimpleSearch retrieved requested number of records
    ok 8 - SimpleSearch for 'book' matched right number of records
    ok 9 - SimpleSearch does not return an error when successful
    ok 10 - SimpleSearch cursor functions
    ok 11 - SimpleSearch handles simple CCL
    ok 12 - SimpleSearch on music publisher number works (bug 8252)
    ok 13 - SimpleSearch on music publisher number works using Identifier-publisher-for-music (bug 8252)
    ok 14 - getRecords keyword search for 'book' matched right number of records
    ok 15 - getRecords returned requested number of records
    ok 16 - getRecords cursor works
    ok 17 - getRecords title search for 'book' matched right number of records
    ok 18 - getRecords title search for 'Australia' matched right number of records
    ok 19 - Simple relevance sorting in getRecords matches old behavior
    ok 20 - Simple ascending author sorting in getRecords matches old behavior
    ok 21 - Simple descending author sorting in getRecords matches old behavior
    ok 22 - Simple ascending publication date sorting in getRecords matches old behavior
    ok 23 - Simple descending publication date sorting in getRecords matches old behavior
    ok 24 - Scan returned requested item
    ok 25 - Scan returned correct number of records matching term
    ok 26 - Built CCL keyword query
    ok 27 - getRecords generated keyword search for 'salud' matched right number of records
    ok 28 - searchResults returns requested number of hits
    ok 29 - Built composed explicit-and CCL keyword query
    ok 30 - getRecords generated composed keyword search for 'salud' explicit-and 'higiene' matched right number of records
    ok 31 - Built composed explicit-or CCL keyword query
    ok 32 - getRecords generated composed keyword search for 'salud' explicit-or 'higiene' matched right number of records
    ok 33 - Built composed implicit-and CCL keyword query
    ok 34 - getRecords generated composed keyword search for 'salud' implicit-and 'higiene' matched right number of records
    ok 35 - Faceted query generated correctly
    ok 36 - Facets not included in query description
    ok 37 - getRecords generated faceted search matched right number of records
    ok 38 - getRecords generated mc-faceted search matched right number of records
    ok 39 - getRecords generated multi-faceted search matched right number of records
    ok 40 - search using index whose name contains "ns" returns expected results (bug 10271)
    ok 41 - Search for 'book' with index set to 'kw' returns 101 hits
    ok 42 - Search for 'kw:book && kw:another' returns 1 hit
    ok 43 - getRecords generated availability-limited search matched right number of records
    not ok 44 - All records have at least one item available

    #   Failed test 'All records have at least one item available'
    #   at t/db_dependent/Search.t line 510.
    #          got: 'false'
    #     expected: 'true'
    ok 45 - getRecords on _ALLRECORDS PQF returned all records
    ok 46 - getRecords PQF author search for Lessig returned proper number of matches
    ok 47 - getRecords CCL author search for Lessig returned proper number of matches
    ok 48 - getRecords CQL author search for Lessig returned proper number of matches
    ok 49 - Weighted query returned correct number of results
    ok 50 - Weighted query returns best match first
    ok 51 - Search for 'medic' returns matches  with automatic truncation on
    ok 52 - Search for 'medic*' returns matches with automatic truncation on
    ok 53 - Search for 'web application' returns one hit with QueryWeightFields on
    ok 54 - Search for 'web "application' returns one hit with QueryWeightFields on (bug 7518)
    ok 55 - Search for 'medic' returns no matches with automatic truncation off
    ok 56 - Search for 'medic*' returns matches with automatic truncation off
    ok 57 - Search for 'pressed' returns matches when stemming (and query weighting) is on
    ok 58 - Search for 'pressed' returns no matches when stemming is off
    ok 59 - buildQuery should add the available part to the query if requested with ccl
    ok 60 - buildQuery should remove the available part from the query
    ok 61 - buildQuery should keep 0 value
    ok 62 - SimpleSearch warns about CCL parsing error with nonsense query
    ok 63 - SimpleSearch returns an error when passed gibberish
    ok 64 - getRecords warns about unknown query type
    ok 65 - getRecords warns when query type is not specified for non-CCL query
    ok 66 - Alternate holdings filled in correctly
    ok 67 - presence of a transit does not block hold request action (bug 10741)
    ok 68 - search for ti:punctuation returned expected number of records
    ok 69 - Warning is raised correctly for invalid tags in MARC::Record
    ok 70 - a record that cannot be parsed by MARC::Record is simply skipped (bug 10684)
Negative repeat count does nothing at /home/vagrant/kohaclone/Koha/QueryParser/Driver/PQF/query_plan.pm line 53.
    ok 71 - Searching for expected term 'Arizona' for exploding
    ok 72 - Advanced search for broader subjects
Negative repeat count does nothing at /home/vagrant/kohaclone/Koha/QueryParser/Driver/PQF/query_plan.pm line 53.
    ok 73 - Searching for expected term 'Arizona' for exploding
    ok 74 - Advanced search for narrower subjects
Negative repeat count does nothing at /home/vagrant/kohaclone/Koha/QueryParser/Driver/PQF/query_plan.pm line 53.
    ok 75 - Searching for expected term 'Arizona' for exploding
    ok 76 - Advanced search for related subjects
Negative repeat count does nothing at /home/vagrant/kohaclone/Koha/QueryParser/Driver/PQF/query_plan.pm line 53.
    ok 77 - Searching for expected term 'Arizona' for exploding
    ok 78 - Advanced search for related subjects and keyword 'history' searches related subjects
    ok 79 - Advanced search for related subjects and keyword 'history' searches for 'history'
Negative repeat count does nothing at /home/vagrant/kohaclone/Koha/QueryParser/Driver/PQF/query_plan.pm line 53.
    ok 80 - Searching for expected term 'Arizona' for exploding
    ok 81 - Order of terms doesn't matter for advanced search
    ok 82 - Order of terms doesn't matter for advanced search
Negative repeat count does nothing at /home/vagrant/kohaclone/Koha/QueryParser/Driver/PQF/query_plan.pm line 53.
    ok 83 - Searching for expected term 'Arizona' for exploding
    ok 84 - Simple search for broader subjects
Negative repeat count does nothing at /home/vagrant/kohaclone/Koha/QueryParser/Driver/PQF/query_plan.pm line 53.
    ok 85 - Searching for expected term 'Arizona' for exploding
    ok 86 - Simple search for narrower subjects
Negative repeat count does nothing at /home/vagrant/kohaclone/Koha/QueryParser/Driver/PQF/query_plan.pm line 53.
    ok 87 - Searching for expected term 'Arizona' for exploding
    ok 88 - Simple search for related subjects
Negative repeat count does nothing at /home/vagrant/kohaclone/Koha/QueryParser/Driver/PQF/query_plan.pm line 53.
    ok 89 - Searching for expected term 'Arizona' for exploding
    ok 90 - Simple search for related subjects and keyword 'history' searches related subjects
    ok 91 - Simple search for related subjects and keyword 'history' searches for 'history'
    ok 92 - use C4::AuthoritiesMarc;
    ok 93 - MARC21 authorities: one hit on mainentry starts with "shakespeare"
    ok 94 - MARC21 authorities: one hit on mainentry starts with "shakespeare" sorted by heading ascending
    ok 95 - MARC21 authorities: one hit on mainentry starts with "shakespeare" sorted by heading descending
    ok 96 - MARC21 authorities: one hit on match contains "沙士北亞威廉姆"
    ok 97 - MARC21 authorities: one hit on LC-card-number contains "99282477"
    ok 98 - MARC21 authorities: one hit on "all" (entire record) contains "professional wrestler"
Negative repeat count does nothing at /home/vagrant/kohaclone/Koha/QueryParser/Driver/PQF/query_plan/node.pm line 102.
    ok 99 - MARC21 authorities: one hit on mainentry starts with "shakespeare" (QP)
Negative repeat count does nothing at /home/vagrant/kohaclone/Koha/QueryParser/Driver/PQF/query_plan/node.pm line 102.
    ok 100 - MARC21 authorities: one hit on mainentry starts with "shakespeare" sorted by heading ascending (QP)
Negative repeat count does nothing at /home/vagrant/kohaclone/Koha/QueryParser/Driver/PQF/query_plan/node.pm line 102.
    ok 101 - MARC21 authorities: one hit on mainentry starts with "shakespeare" sorted by heading descending (QP)
Negative repeat count does nothing at /home/vagrant/kohaclone/Koha/QueryParser/Driver/PQF/query_plan/node.pm line 102.
    ok 102 - MARC21 authorities: one hit on match contains "沙士北亞威廉姆" (QP)
    ok 103 - Can do a search that retrieves an over-large bib record (bug 11096)
    ok 104 - Able to render the title for over-large bib record (bug 11096)
    ok 105 - Over-large bib record has the correct biblionumber (bug 11096)
    ok 106 - Able to render the notes field for over-large bib record (bug 11096)
    ok 107 - 'notforloancount' defined in searchResults output (Bug 12419)
    ok 108 - 'notforloancount' == 2 (Bug 12419)
    ok 109 - got warning instead of crash when attempting to run invalid query (bug 9578)
    ok 110 - _get_facets_data_from_record doesn't count 100$z (Bug 12788)
    ok 111 - _get_facets_data_from_record correctly counts author facet twice
    ok 112 - _get_facets_info returns the correct data
    # Looks like you failed 1 test of 112.
not ok 1 - MARC21 + DOM

#   Failed test 'MARC21 + DOM'
#   at t/db_dependent/Search.t line 973.
# Subtest: UNIMARC + DOM
    1..14
    ok 1 - use C4::Search;
    ok 2 - UNIMARC title search
    ok 3 - UNIMARC target audience = u
    ok 4 - UNIMARC target audience = k
    ok 5 - UNIMARC target audience = m
    ok 6 - UNIMARC generic item index (bug 10037)
    ok 7 - use C4::AuthoritiesMarc;
    ok 8 - UNIMARC authorities: hits on mainentry contains "wil"
    ok 9 - UNIMARC authorities: hits on match contains "wil"
    ok 10 - UNIMARC authorities: hits on mainentry contains "michel"
    ok 11 - UNIMARC authorities: hits on mainmainentry = "valley"
    ok 12 - UNIMARC authorities: no hits on mainmainentry = "vall"
    ok 13 - UNIMARC authorities: hits on any starts with "jean"
    ok 14 - _get_facets_info returns the correct data
ok 2 - UNIMARC + DOM
# Looks like you failed 1 test of 2.
Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/2 subtests 

Test Summary Report
-------------------
t/db_dependent/Search.t (Wstat: 256 Tests: 2 Failed: 1)
  Failed test:  1
  Non-zero exit status: 1
Files=1, Tests=2, 10 wallclock secs ( 0.03 usr  0.02 sys +  5.47 cusr  1.24 csys =  6.76 CPU)
Result: FAIL
Comment 33 Martin Renvoize 2019-08-14 12:47:42 UTC
On a fresh koha-testing-docker the tests are passing for me.. I'm wondering if there's something data related on your system causing some fun and games perhaps Hayley?
Comment 34 Hayley Pelham 2019-08-14 21:47:18 UTC
Created attachment 92202 [details] [review]
Bug 11677: Use zebra not-onloan-count index for availability limit

This uses the work from bug 18098 to use the count of not-onloan items to
determine availability.

It keeps the allrecords,AlwaysMatches='' portion of the query so it still catches
records with the tested indexed missing.
These shouldn't exist except in the case of missing data, but kept for future use as well as
it has no negative impact and minimizes the change made.

Note that availability in this sense means not check out and not lost. Transfers, holds, and not for loan statuses
should probably have indexes built in the same way as bug 18098 (there are related FIXMEs in the code)

This updates the export_records file used for testing - adding three records with a not-onloan-count (999$x) of 1,
but marking the item on the third record lost to prevent its return

To test:
   1 - make sure SearchEngine pref is set to 'Zebra'
   2 - Do a search on the OPAC that returns some records
   3 - Find a record in those results with more than one 1 item, and checkout 1 leaving 1 (or more) available
   4 - Find another record with only 1 item, check that item out
   5 - Limit opac search to available items - both records disappear
   6 - Apply patch
   7 - Restart all the things
   8 - Repeat search
   9 - First record should show, but no second
  10 - Mark other items of first record as lost, return items on second record
  11 - Repeat search, first record should not be in results, second should
  12 - prove -v t/db_dependent/Search.t

Signed-off-by: Hayley Mapley <hayleymapley@catalyst.net.nz>
Comment 35 Hayley Pelham 2019-08-14 21:48:26 UTC
I thought I'd reset my database before testing Martin, but it looks like I didn't!
Sorry about that :)
Comment 36 Martin Renvoize 2019-08-16 09:51:52 UTC
(In reply to Hayley Mapley from comment #35)
> I thought I'd reset my database before testing Martin, but it looks like I
> didn't!
> Sorry about that :)

Phew, no problems at all.. I've done the same many a time ;)
Comment 37 Tomás Cohen Arazi 2019-08-19 15:23:35 UTC
Created attachment 92351 [details] [review]
Bug 11677: Use zebra not-onloan-count index for availability limit

This uses the work from bug 18098 to use the count of not-onloan items to
determine availability.

It keeps the allrecords,AlwaysMatches='' portion of the query so it still catches
records with the tested indexed missing.
These shouldn't exist except in the case of missing data, but kept for future use as well as
it has no negative impact and minimizes the change made.

Note that availability in this sense means not check out and not lost. Transfers, holds, and not for loan statuses
should probably have indexes built in the same way as bug 18098 (there are related FIXMEs in the code)

This updates the export_records file used for testing - adding three records with a not-onloan-count (999$x) of 1,
but marking the item on the third record lost to prevent its return

To test:
   1 - make sure SearchEngine pref is set to 'Zebra'
   2 - Do a search on the OPAC that returns some records
   3 - Find a record in those results with more than one 1 item, and checkout 1 leaving 1 (or more) available
   4 - Find another record with only 1 item, check that item out
   5 - Limit opac search to available items - both records disappear
   6 - Apply patch
   7 - Restart all the things
   8 - Repeat search
   9 - First record should show, but no second
  10 - Mark other items of first record as lost, return items on second record
  11 - Repeat search, first record should not be in results, second should
  12 - prove -v t/db_dependent/Search.t

Signed-off-by: Hayley Mapley <hayleymapley@catalyst.net.nz>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Comment 38 Martin Renvoize 2019-08-20 14:02:25 UTC
Nice work!

Pushed to master for 19.11.00
Comment 39 Fridolin Somers 2019-09-05 13:57:04 UTC
Pushed to 19.05.x for 19.05.04
Comment 40 Lucas Gass 2019-09-05 21:55:53 UTC
backported to 18.11.x for 18.11.10
Comment 41 Christopher Brannon 2019-09-05 22:20:45 UTC
(In reply to Lucas Gass from comment #40)
> backported to 18.11.x for 18.11.10

+1
Comment 42 Magnus Enger 2020-02-12 10:04:25 UTC
Did this fix the "limit to available" for folks? I still can't seem to make it work on 19.05.07 (with Zebra). I do an advanced search for an itemtype and get lots of results, some on loan and some not. Then I click on the "Limit to currently available items" and get zero results. Same result after reindexing.
Comment 43 Magnus Enger 2020-02-12 10:13:35 UTC
(In reply to Magnus Enger from comment #42)
> Did this fix the "limit to available" for folks? I still can't seem to make
> it work on 19.05.07 (with Zebra). I do an advanced search for an itemtype
> and get lots of results, some on loan and some not. Then I click on the
> "Limit to currently available items" and get zero results. Same result after
> reindexing.

Looks like I missed the update to biblios/etc/bib1.att from bug 18098 during an upgrade. Sorry for the noise!
Comment 44 Katrin Fischer 2020-04-18 23:22:13 UTC
*** Bug 16970 has been marked as a duplicate of this bug. ***