Bug 13420

Summary: Holdings table sorting on volume information incorrect
Product: Koha Reporter: Nicole C. Engard <nengard>
Component: CatalogingAssignee: Jonathan Druart <jonathan.druart>
Status: CLOSED FIXED QA Contact: Josef Moravec <josef.moravec>
Severity: major    
Priority: P5 - low CC: 1joynelson, aleisha, bklein, emjhet, horganl, jonathan.druart, josef.moravec, jschmidt, katrin.fischer, kholten, kyle, lucas, m.de.rooy, marjorie.barry-vila, martin.renvoize, wizzyrea
Version: Main   
Hardware: All   
OS: All   
See Also: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12375
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=5594
Change sponsored?: --- Patch complexity: ---
Documentation contact: Documentation submission:
Text to go in the release notes:
Version(s) released in:
20.05.00, 19.11.04, 19.05.09
Bug Depends on: 12375    
Bug Blocks: 28171    
Attachments: opac
staff
Screenshot of bug in context in the staff view
Bug 13420: Sort "Serial enumeration" by published date
Bug 13420: Fallback to the previous behavior if published date is not used
Bug 13420: Sort "Serial enumeration" by published date
Bug 13420: Fallback to the previous behavior if published date is not used
Bug 13420: Sort "Serial enumeration" by published date
Bug 13420: Fallback to the previous behavior if published date is not used
Bug 13420: (QA follow-up): Fix typo itemdate -> itemdata

Description Nicole C. Engard 2014-12-09 16:15:13 UTC
Created attachment 34239 [details]
opac

See attached. In the staff client and OPAC when you short on volume things are not sorted the way the user would expect.  19 is coming after 160
Comment 1 Nicole C. Engard 2014-12-09 16:15:31 UTC
Created attachment 34240 [details]
staff
Comment 2 Aleisha Amohia 2016-01-21 23:28:39 UTC
How do I reproduce this? Is it still valid?
Comment 3 Nicole C. Engard 2016-01-27 19:18:56 UTC

*** This bug has been marked as a duplicate of bug 12375 ***
Comment 4 Nicole C. Engard 2016-01-27 19:20:37 UTC
This is still valid.
Comment 5 Aleisha Amohia 2016-01-27 22:39:47 UTC
Can you do a screenshot of the full page please so I can see where this is and how to reproduce it
Comment 6 Karl Holten 2016-02-02 19:18:15 UTC
Created attachment 47566 [details]
Screenshot of bug in context in the staff view

While the duplicate bug has been marked as fixed, I am not entirely clear on if that bug will address this issue, so I will update this ticket. Attached is a screenshot of the bug in context in the staff view. The OPAC view sorts in a similar way.
Comment 7 Kyle M Hall 2016-02-11 14:17:07 UTC
This issue is that the volume column is being sorted as a string, rather than a number. This does and does not make sense. The volume field is not strictly a number, but is a general text field. Some libraries may put "123" in there, or "vol 123" or as is the case for Karl, "v. 123".

The quick hack solution would be to pad out all the volume numbers to the same length in the volume field itself. This is a very sub-optimal solution.

I think the optimal solution would be to take each volume field and pad out each number in the string to 10 or so digits. We can then take the padded version and put it in a hidden column that is used for sorting the visible column.

The regex s/([0-9]+)/sprintf('%010d',$1)/ge; should take care of the padding.
Comment 8 Kyle M Hall 2016-02-11 14:30:28 UTC
Another option, considering this is a serials column, would be to use the serial sequence columns x y and x directly, instead of the concatenated version. The need for padding would still apply though.
Comment 9 Karl Holten 2016-03-01 18:55:17 UTC
I'd think the most desirable solution would be use padded X Y and Z columns on the backend to conduct the sort, and have the nicely formatted version display.
Comment 10 emjhet 2018-06-20 22:41:32 UTC
This is a pet peeve of mine. If a missed serial is donated at a later date, I never get to see my serials in sequential order, because sorting by Vol replicates what is seen above, while sorting by call number alphabetizes them so all Aprils (regardless of year) are before all the Novembers. The only way they are in order is if my barcodes are in order; that only happens if no issues are missed or go walking.
Comment 11 Brian 2018-12-15 18:29:54 UTC
Any more progress?

Carnegie Team
Comment 12 Marjorie Barry-Vila 2019-03-01 20:06:43 UTC
+1
Comment 13 Jonathan Druart 2019-03-05 16:03:05 UTC
Would it make sense to sort by published dates?
Comment 14 Laura Horgan 2019-03-13 17:45:44 UTC
We would be interested in a sort by published date as we do not enter volumes or issues as finding that information too way to long for our staff.  Fewer items are claimed these days and very few people ever come to us looking for a specific volume/issue of a periodical.
Comment 15 Jonathan Druart 2019-04-29 03:01:11 UTC
Created attachment 88995 [details] [review]
Bug 13420: Sort "Serial enumeration" by published date

So far the sorting on volume information is incorrect. It sorts by the
string displayed in the cell, which may not have any logics.

A better solution would be to use Kyle's suggestion, see comments 7, 8,
but it will need much more work.

This patch suggests to use the publication date to sort this column.
Note: In the code there is a switch depending on the existence of
items.publisheddate, but I do not think it is valid, this valid should
always exist if the item is received (did I miss something?)

Test plan:
You need to have different existing subscription, using different
numbering.
On the detail page of the bibliographic record you should be able to
sort the serials (tab "Holdings") by "Serial enumeration".
The sort will now use the publication dates.
Comment 16 Katrin Fischer 2019-04-29 05:59:31 UTC
Libraries not using the serials module still use the enumchron fields. So a fallback to the current behaviour is needed, even if not perfect.
Comment 17 Liz Rea 2019-05-06 16:40:23 UTC
This is actually a really hard bug to fix because that field isn't uniform across libraries, and it's human input (sometimes) as well. It can be just numeric, alphanumeric, or only words.

I don't know how you even could universally fix this sorting problem, short of making it so people could choose the type of sort they want on the column.

Liz
Comment 18 Katrin Fischer 2019-07-31 17:34:18 UTC
I like the idea of a configurable sort for now with the option to add a more complicated fix using the serial pattern variables later on.

So maybe:
- publicationdate with fallback to alphabetic if empty
- strictly alphabetic
Comment 19 Jonathan Druart 2019-08-06 13:58:19 UTC
Created attachment 92026 [details] [review]
Bug 13420: Fallback to the previous behavior if published date is not used
Comment 20 Jonathan Druart 2019-08-06 13:58:52 UTC
(In reply to Katrin Fischer from comment #18)
> I like the idea of a configurable sort for now with the option to add a more
> complicated fix using the serial pattern variables later on.
> 
> So maybe:
> - publicationdate with fallback to alphabetic if empty

The last patch implements that suggestion.
Comment 21 Katrin Fischer 2020-01-06 13:28:56 UTC
Should we change this to 'Needs Signoff' to get some opinions?
Comment 22 Jonathan Druart 2020-01-06 21:23:55 UTC
(In reply to Katrin Fischer from comment #21)
> Should we change this to 'Needs Signoff' to get some opinions?

Yes, done.
Comment 23 Martin Renvoize 2020-01-27 19:19:05 UTC
Created attachment 97994 [details] [review]
Bug 13420: Sort "Serial enumeration" by published date

So far the sorting on volume information is incorrect. It sorts by the
string displayed in the cell, which may not have any logics.

A better solution would be to use Kyle's suggestion, see comments 7, 8,
but it will need much more work.

This patch suggests to use the publication date to sort this column.
Note: In the code there is a switch depending on the existence of
items.publisheddate, but I do not think it is valid, this valid should
always exist if the item is received (did I miss something?)

Test plan:
You need to have different existing subscription, using different
numbering.
On the detail page of the bibliographic record you should be able to
sort the serials (tab "Holdings") by "Serial enumeration".
The sort will now use the publication dates.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 24 Martin Renvoize 2020-01-27 19:19:09 UTC
Created attachment 97995 [details] [review]
Bug 13420: Fallback to the previous behavior if published date is not used

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 25 Martin Renvoize 2020-01-27 19:19:37 UTC
This works as described and seems a suitable interim solution to me.. Signing off.
Comment 26 Josef Moravec 2020-02-05 21:14:49 UTC
Created attachment 98511 [details] [review]
Bug 13420: Sort "Serial enumeration" by published date

So far the sorting on volume information is incorrect. It sorts by the
string displayed in the cell, which may not have any logics.

A better solution would be to use Kyle's suggestion, see comments 7, 8,
but it will need much more work.

This patch suggests to use the publication date to sort this column.
Note: In the code there is a switch depending on the existence of
items.publisheddate, but I do not think it is valid, this valid should
always exist if the item is received (did I miss something?)

Test plan:
You need to have different existing subscription, using different
numbering.
On the detail page of the bibliographic record you should be able to
sort the serials (tab "Holdings") by "Serial enumeration".
The sort will now use the publication dates.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Comment 27 Josef Moravec 2020-02-05 21:14:54 UTC
Created attachment 98512 [details] [review]
Bug 13420: Fallback to the previous behavior if published date is not used

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Comment 28 Josef Moravec 2020-02-05 21:14:58 UTC
Created attachment 98513 [details] [review]
Bug 13420: (QA follow-up): Fix typo itemdate -> itemdata

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Comment 29 Martin Renvoize 2020-02-06 12:37:50 UTC
Nice work everyone!

Pushed to master for 20.05
Comment 30 Joy Nelson 2020-03-05 00:07:15 UTC
Pushed to 19.11.x for 19.11.04
Comment 31 Lucas Gass 2020-03-09 17:49:54 UTC
backported to 19.05.x for 19.05.09