Bug 15331 - Sort order breaks at offset=1000 (both OPAC and Intranet)
Summary: Sort order breaks at offset=1000 (both OPAC and Intranet)
Status: RESOLVED DUPLICATE of bug 7041
Alias: None
Product: Koha
Classification: Unclassified
Component: Searching (show other bugs)
Version: Main
Hardware: All All
: P5 - low enhancement with 30 votes (vote)
Assignee: Galen Charlton
QA Contact: Testopia
URL:
Keywords:
Depends on: 7041
Blocks:
  Show dependency treegraph
 
Reported: 2015-12-08 16:48 UTC by Barton Chittenden
Modified: 2020-01-13 05:27 UTC (History)
7 users (show)

See Also:
Change sponsored?: ---
Patch complexity: ---
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Barton Chittenden 2015-12-08 16:48:02 UTC
To re-create:

1) Go to the advanced search page, in either Staff Client or OPAC.
2) Run a search which will return many thousands of results, sort by A-Z
3) Go to the page prior to offset=1000 (i.e. page 50 if you have 20 results/page or 40 if you have 25 results/page). This page will be properly sorted alphabetically.
5) Go to the next page. This will *not* be sorted alphabetically.

Example:

http://catalog.bywatersolutions.com/cgi-bin/koha/opac-search.pl?&limit=mc-itype%2Cphr%3ABOOK&offset=980&sort_by=title_az

The results are

Amazing armored animals / 
Amazing bats / 
Amazing birds / 
The amazing bone / 
The amazing book of bird records : the largest, the smallest, the fastest, and many more! / 
Amazing buildings / 
Amazing buildings / 
Amazing dinosaur facts / 
Amazing flying machines / 
Amazing grace / 
Amazing grace / 
The amazing impossible Erie Canal / 
Amazing Leonardo da Vinci inventions you can build yourself : learn some hands-on history! / 
The amazing life of Benjamin Franklin / 
The amazing life of birds : (the twenty-day puberty journal of Duane Homer Leech) / 
An amazing machine / 
Amazing magic tricks : apprentice level / 
Amazing magic tricks : master level / 
Amazing mammals / 
The Amazing Maurice and his educated rodents / 

The next titles in the catalog should be

Amazing mazes : mind bending mazes for ages 6-60 /
The amazing Mrs. Pollifax 
The amazing odorous adventures of Stinky Dog /

Instead, we get

http://catalog.bywatersolutions.com/cgi-bin/koha/opac-search.pl?&limit=mc-itype%2Cphr%3ABOOK&offset=1000&sort_by=title_az

The history of Herodotus Thucydides. 
Hippocratic writings. 
The day after Roswell / 
The demon-haunted world : science as a candle in the dark / 
The UFO phenomenon / 
Champ, beyond the legend (updated edition) / 
Area 51 : the dreamland chronicles / 
UFO crash at Roswell : the genesis of a modern myth /  
Chaos : making a new science / 
The black swan : the impact of the highly improbable / 
PC annoyances : how to fix the most annoying things about your personal computer / 
PC help desk in a book : the do-it- yourself guide to PC troubleshooting and repair /  
The little Mac book / 
Where wizards stay up late : the origins of the Internet / 
Web design tools and techniques / 
Degunking your email, spam, and viruses / 
IPhone and iPad apps marketing : secrets to selling your iPhone and iPad apps / 
Office 2007 all-in-one desk reference for dummies / 
Windows 95 secrets /
Comment 1 Frédéric Demians 2015-12-08 17:22:10 UTC
Take a look at bug 7041
Comment 2 David Cook 2015-12-08 22:17:07 UTC
(In reply to Frédéric Demians from comment #1)
> Take a look at bug 7041

You might also take a look at "sortmax" at http://www.indexdata.com/zebra/doc/zebra-cfg.html
Comment 3 Nick Clemens 2016-03-09 02:06:26 UTC
Is this valid then, or does sortmax/bug 7041 solve this?
Comment 4 David Cook 2016-04-11 04:00:14 UTC
(In reply to Nick Clemens from comment #3)
> Is this valid then, or does sortmax/bug 7041 solve this?

sortmax would allow you to sort more records, but it'll always be a limitation, I think. I wonder if there's a way to specify sort all...
Comment 5 Katrin Fischer 2020-01-08 22:16:05 UTC
Reading through bug 7041 it looks like there is a configuration setting to change the max records to sort - closing as a duplicate.

*** This bug has been marked as a duplicate of bug 7041 ***
Comment 6 Patrick 2020-01-09 09:50:37 UTC
On review can I ask if there is a difference between 7041 & 15331 in that: 

What is being requested here is a "sort all" parameter - that is the system will sort all results in the set whether it is 10 or 100,000.

Where as 7041 is a specified number and after that number the results will be no longer sorted.

Potentially this would cause a performance hit on some searches, but that would be expected.
Comment 7 Katrin Fischer 2020-01-09 10:09:46 UTC
I fear the performance hit would be too big to justify a change to a high number by default. What we could do is document this option/behaviour better, so libraries can choose to take the hit.
Comment 8 David Cook 2020-01-13 05:27:40 UTC
(In reply to Katrin Fischer from comment #7)
> I fear the performance hit would be too big to justify a change to a high
> number by default. What we could do is document this option/behaviour
> better, so libraries can choose to take the hit.

I agree with Katrin. Koha is used by libraries big and small, so I think it's a better idea to be more conservative by default. (That said, it would be interesting if someone did some benchmarks on this one. Perhaps 1000 is an arbitrary number. We don't have numbers on the actual performance degradation at present I think.)

I am also curious about the scenarios where people are getting search results higher than 1000. Back at university, as a student, I think librarians and professors would've told us that our searches were too broad if we were returning that many results. (That said, maybe you are doing a search for all science fiction and want to search through the results alphabetically, as you can't quite remember exactly what you're looking for. That seems reasonable too.)