Bug 19316 - Items should have sort1 and sort2 fields
Summary: Items should have sort1 and sort2 fields
Status: Failed QA
Alias: None
Product: Koha
Classification: Unclassified
Component: Database (show other bugs)
Version: Main
Hardware: All All
: P5 - low enhancement
Assignee: Matt Blenkinsop
QA Contact: Nick Clemens (kidclamp)
URL:
Keywords: release-notes-needed
: 32897 (view as bug list)
Depends on:
Blocks:
 
Reported: 2017-09-14 13:46 UTC by Nick Clemens (kidclamp)
Modified: 2024-08-19 16:25 UTC (History)
6 users (show)

See Also:
Change sponsored?: ---
Patch complexity: ---
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:
Circulation function:


Attachments
Bug 19316: Add sort1 and sort2 to items and deleteditems tables (3.90 KB, patch)
2023-08-30 14:09 UTC, Matt Blenkinsop
Details | Diff | Splinter Review
Bug 19316: Add DBIC files (2.94 KB, patch)
2023-08-30 14:09 UTC, Matt Blenkinsop
Details | Diff | Splinter Review
Bug 19316: Add sort1 and sort2 fields to interface and allow editing (17.19 KB, patch)
2023-08-30 14:09 UTC, Matt Blenkinsop
Details | Diff | Splinter Review
Bug 19316: Update items api definition (1.04 KB, patch)
2023-08-30 14:09 UTC, Matt Blenkinsop
Details | Diff | Splinter Review
Bug 19316: (follow-up) Tidy Items.pm (17.84 KB, patch)
2023-08-30 14:09 UTC, Matt Blenkinsop
Details | Diff | Splinter Review
Bug 19316: Add sort1 and sort2 to items and deleteditems tables (3.90 KB, patch)
2024-05-07 15:55 UTC, Matt Blenkinsop
Details | Diff | Splinter Review
Bug 19316: Add DBIC files (2.01 KB, patch)
2024-05-07 15:55 UTC, Matt Blenkinsop
Details | Diff | Splinter Review
Bug 19316: Add sort1 and sort2 fields to interface and allow editing (17.47 KB, patch)
2024-05-07 15:55 UTC, Matt Blenkinsop
Details | Diff | Splinter Review
Bug 19316: Update items api definition (1.04 KB, patch)
2024-05-07 15:55 UTC, Matt Blenkinsop
Details | Diff | Splinter Review
Bug 19316: Add CSRF protection (3.98 KB, patch)
2024-05-10 15:10 UTC, Matt Blenkinsop
Details | Diff | Splinter Review
Bug 19316: Add CSRF protection (4.01 KB, patch)
2024-05-10 18:20 UTC, Baptiste Bayche
Details | Diff | Splinter Review
Bug 19316: Add sort1 and sort2 to items and deleteditems tables (3.90 KB, patch)
2024-07-24 10:19 UTC, Matt Blenkinsop
Details | Diff | Splinter Review
Bug 19316: Add DBIC files (2.07 KB, patch)
2024-07-24 10:19 UTC, Matt Blenkinsop
Details | Diff | Splinter Review
Bug 19316: Add sort1 and sort2 fields to interface and allow editing (25.76 KB, patch)
2024-07-24 10:19 UTC, Matt Blenkinsop
Details | Diff | Splinter Review
Bug 19316: Update items api definition (1.10 KB, patch)
2024-07-24 10:19 UTC, Matt Blenkinsop
Details | Diff | Splinter Review
Bug 19316: Add CSRF protection (4.03 KB, patch)
2024-07-24 10:20 UTC, Matt Blenkinsop
Details | Diff | Splinter Review
Bug 19316: Add sort1 and sort2 to items and deleteditems tables (3.94 KB, patch)
2024-07-24 15:57 UTC, Nick Clemens (kidclamp)
Details | Diff | Splinter Review
Bug 19316: Add DBIC files (2.11 KB, patch)
2024-07-24 15:57 UTC, Nick Clemens (kidclamp)
Details | Diff | Splinter Review
Bug 19316: Add sort1 and sort2 fields to interface and allow editing (25.80 KB, patch)
2024-07-24 15:58 UTC, Nick Clemens (kidclamp)
Details | Diff | Splinter Review
Bug 19316: Update items api definition (1.14 KB, patch)
2024-07-24 15:58 UTC, Nick Clemens (kidclamp)
Details | Diff | Splinter Review
Bug 19316: Add CSRF protection (4.07 KB, patch)
2024-07-24 15:58 UTC, Nick Clemens (kidclamp)
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Nick Clemens (kidclamp) 2017-09-14 13:46:45 UTC
Librarians love data, like for real, they can't get enough. 

We often find that libraries want extra information in their items table, things that don't fit or don't belong in any of the existing columns.  

Borrowers have sort1 and sort2 for this kind of information - we should add analogous fields to the items table
Comment 1 Katrin Fischer 2023-02-26 11:07:43 UTC
*** Bug 32897 has been marked as a duplicate of this bug. ***
Comment 2 Matt Blenkinsop 2023-08-30 14:09:45 UTC
Created attachment 154963 [details] [review]
Bug 19316: Add sort1 and sort2 to items and deleteditems tables

This patch adds to new fields to the items table to allow input for additional data into item records to help with sorting

Sponsored-by: PTFS Europe
Comment 3 Matt Blenkinsop 2023-08-30 14:09:47 UTC
Created attachment 154964 [details] [review]
Bug 19316: Add DBIC files
Comment 4 Matt Blenkinsop 2023-08-30 14:09:50 UTC
Created attachment 154965 [details] [review]
Bug 19316: Add sort1 and sort2 fields to interface and allow editing

This patch adds the new sort1 and sort2 fields to the interface so that they can be viewed and edited

Test plan:
1) Apply patch and reset_all
2) In the catalog, search for a record with at least one item
3) In the Holdings table, click on the barcode for an item. You should now be on /cgi-bin/koha/catalogue/moredetail.pl
4) Scroll down and observe that there is now a section called "Sort fields" with two fields, Sort 1 and Sort 2.
5) Enter some text in the textarea for Sort 1 (max 50 characters) and click Update
6) The page will refresh and your text should still be visible. Check that item in the database and the text should be stored in the sort1 column.
7) Repeat steps 5-6 for Sort 2.
8) Copy the barcode and navigate to Cataloging > Batch item modification
9) Enter the barcode in the textarea under "Or scan items one by one" (You can add multiple bar codes here if you wish)
10) Fill in the Required values (Home library, Current library, Koha item type)
11) In "Other attributes" there should be two fields, Sort 1 and Sort 2
12) Enter some text into both these fields
13) Click save and a job will be enqueued
14) Click "view detail of the enqueued job"
15) A message should be shown on completion that says "X item(s) modified (with Y field(s) modified)." X should match the number of barcodes that you entered into the batch mod and Y should equal X x 2
16) Check the item either in the interface or the database and the text you entered should now be stored and visible
17) Repeat steps 8 - 14 but this time only fill in Sort 1
18) This time X should equal the number of barcodes that you entered into the batch mod but Y should equal X as only one field was filled in.
19) Again, the text you entered should now be stored on the item record
Comment 5 Matt Blenkinsop 2023-08-30 14:09:52 UTC
Created attachment 154966 [details] [review]
Bug 19316: Update items api definition
Comment 6 Matt Blenkinsop 2023-08-30 14:09:54 UTC
Created attachment 154967 [details] [review]
Bug 19316: (follow-up) Tidy Items.pm

Items.pm kept failing the QA script despite all new lines having been tidied with perltidy. There must be a larger tidying issue in the file that is affecting all the new lines so I've tidied the whole file and added a separate commit.
Comment 7 Matt Blenkinsop 2023-08-30 14:10:59 UTC
If the follow-up commit needs adding to its own bug then let me know and I'll happily do so, I just couldn't get the QA script to pass for that file without tidying the whole file
Comment 8 David Cook 2023-08-31 01:04:38 UTC
At a glance, it looks like this data could leak out to the public through the API. How might that be prevented?
Comment 9 David Cook 2023-08-31 01:16:21 UTC
(In reply to David Cook from comment #8)
> At a glance, it looks like this data could leak out to the public through
> the API. How might that be prevented?

Then again... while it's available via REST API as exposed by the OPAC website... it looks like you need to be a logged in staff user to see it. 

If an API consumer is showing items, hopefully they're using a public endpoint like http://kohadev.mydnsname.org:8080/api/v1/public/biblios/29/items 

So maybe it's fine...

http://kohadev.mydnsname.org:8080/api/v1/items/73

{"error":"Authorization failure. Missing required permission(s).","required_permissions":{"catalogue":"1"}}
Comment 10 Matt Blenkinsop 2023-09-14 16:16:28 UTC
I think the staff permissions should be sufficient protection, this is currently the case on the items table
Comment 11 Martin Renvoize 2024-02-29 07:33:07 UTC
What was the conclusion here in the end?  Do we need a rebase to resume testing or were we considering using additional fields as an alternative..  and David, were you happy with the permissions response?
Comment 12 David Cook 2024-02-29 22:34:42 UTC
(In reply to Martin Renvoize from comment #11)
> and David, were you happy with the permissions response?

More or less heh.

I think we have some significant design flaws with the REST API (particularly in terms of admin API vs public API), but that's out of scope for this one I think.
Comment 13 Matt Blenkinsop 2024-05-07 15:55:26 UTC
Created attachment 166298 [details] [review]
Bug 19316: Add sort1 and sort2 to items and deleteditems tables

This patch adds to new fields to the items table to allow input for additional data into item records to help with sorting

Sponsored-by: PTFS Europe
Comment 14 Matt Blenkinsop 2024-05-07 15:55:28 UTC
Created attachment 166299 [details] [review]
Bug 19316: Add DBIC files
Comment 15 Matt Blenkinsop 2024-05-07 15:55:31 UTC
Created attachment 166300 [details] [review]
Bug 19316: Add sort1 and sort2 fields to interface and allow editing

This patch adds the new sort1 and sort2 fields to the interface so that they can be viewed and edited

Test plan:
1) Apply patch and reset_all
2) In the catalog, search for a record with at least one item
3) In the Holdings table, click on the barcode for an item. You should now be on /cgi-bin/koha/catalogue/moredetail.pl
4) Scroll down and observe that there is now a section called "Sort fields" with two fields, Sort 1 and Sort 2.
5) Enter some text in the textarea for Sort 1 (max 50 characters) and click Update
6) The page will refresh and your text should still be visible. Check that item in the database and the text should be stored in the sort1 column.
7) Repeat steps 5-6 for Sort 2.
8) Copy the barcode and navigate to Cataloging > Batch item modification
9) Enter the barcode in the textarea under "Or scan items one by one" (You can add multiple bar codes here if you wish)
10) Fill in the Required values (Home library, Current library, Koha item type)
11) In "Other attributes" there should be two fields, Sort 1 and Sort 2
12) Enter some text into both these fields
13) Click save and a job will be enqueued
14) Click "view detail of the enqueued job"
15) A message should be shown on completion that says "X item(s) modified (with Y field(s) modified)." X should match the number of barcodes that you entered into the batch mod and Y should equal X x 2
16) Check the item either in the interface or the database and the text you entered should now be stored and visible
17) Repeat steps 8 - 14 but this time only fill in Sort 1
18) This time X should equal the number of barcodes that you entered into the batch mod but Y should equal X as only one field was filled in.
19) Again, the text you entered should now be stored on the item record
Comment 16 Matt Blenkinsop 2024-05-07 15:55:33 UTC
Created attachment 166301 [details] [review]
Bug 19316: Update items api definition
Comment 17 Matt Blenkinsop 2024-05-07 15:56:15 UTC
Rebased and managed to fix the QA tests without tidying the whole Items.pm file
Comment 18 Baptiste Bayche 2024-05-09 21:07:43 UTC
Hi,
I realized the test plan, but at the step 15) I have a message that say "No items modified.". The good item is found, but is not modified.
Comment 19 Matt Blenkinsop 2024-05-10 15:10:15 UTC
Created attachment 166578 [details] [review]
Bug 19316: Add CSRF protection
Comment 20 Matt Blenkinsop 2024-05-10 15:10:48 UTC
I've rebased and added missing CRSF protection. Test plan works fine for me now, would you be able to try again? Thanks!
Comment 21 Baptiste Bayche 2024-05-10 18:17:16 UTC
Hi,
It works perfectly, thank you!
Comment 22 Baptiste Bayche 2024-05-10 18:20:20 UTC
Created attachment 166593 [details] [review]
Bug 19316: Add CSRF protection

Signed-off-by: baptiste <baptiste.bayche@inlibro.com>
Comment 23 Kyle M Hall 2024-07-19 18:13:48 UTC
I'm getting some odd behavior:

t/db_dependent/api/v1/items/bundled_items.t (Wstat: 512 (exited 2) Tests: 1 Failed: 1)
  Failed test:  1
  Non-zero exit status: 2
  Parse errors: Bad plan.  You planned 2 tests but ran 1.

***************************************************************************
DBIx::Class::Relationship::ManyToMany::many_to_many():
***************************************************************************
The many-to-many relationship 'bundle_items' is trying to create a utility method
called set_bundle_items.
This will completely overwrite one such already existing method on class
Koha::Schema::Result::Item.

You almost certainly want to rename your method or the many-to-many
relationship, as the functionality of the original method will not be
accessible anymore.

To disable this warning set to a true value the environment variable
DBIC_OVERWRITE_HELPER_METHODS_OK
Comment 24 Matt Blenkinsop 2024-07-24 10:19:50 UTC
Created attachment 169466 [details] [review]
Bug 19316: Add sort1 and sort2 to items and deleteditems tables

This patch adds to new fields to the items table to allow input for additional data into item records to help with sorting

Sponsored-by: PTFS Europe
Comment 25 Matt Blenkinsop 2024-07-24 10:19:53 UTC
Created attachment 169467 [details] [review]
Bug 19316: Add DBIC files

Signed-off-by: baptiste <baptiste.bayche@inlibro.com>
Comment 26 Matt Blenkinsop 2024-07-24 10:19:55 UTC
Created attachment 169468 [details] [review]
Bug 19316: Add sort1 and sort2 fields to interface and allow editing

This patch adds the new sort1 and sort2 fields to the interface so that they can be viewed and edited

Test plan:
1) Apply patch and reset_all
2) In the catalog, search for a record with at least one item
3) In the Holdings table, click on the barcode for an item. You should now be on /cgi-bin/koha/catalogue/moredetail.pl
4) Scroll down and observe that there is now a section called "Sort fields" with two fields, Sort 1 and Sort 2.
5) Enter some text in the textarea for Sort 1 (max 50 characters) and click Update
6) The page will refresh and your text should still be visible. Check that item in the database and the text should be stored in the sort1 column.
7) Repeat steps 5-6 for Sort 2.
8) Copy the barcode and navigate to Cataloging > Batch item modification
9) Enter the barcode in the textarea under "Or scan items one by one" (You can add multiple bar codes here if you wish)
10) Fill in the Required values (Home library, Current library, Koha item type)
11) In "Other attributes" there should be two fields, Sort 1 and Sort 2
12) Enter some text into both these fields
13) Click save and a job will be enqueued
14) Click "view detail of the enqueued job"
15) A message should be shown on completion that says "X item(s) modified (with Y field(s) modified)." X should match the number of barcodes that you entered into the batch mod and Y should equal X x 2
16) Check the item either in the interface or the database and the text you entered should now be stored and visible
17) Repeat steps 8 - 14 but this time only fill in Sort 1
18) This time X should equal the number of barcodes that you entered into the batch mod but Y should equal X as only one field was filled in.
19) Again, the text you entered should now be stored on the item record

Signed-off-by: baptiste <baptiste.bayche@inlibro.com>
Comment 27 Matt Blenkinsop 2024-07-24 10:19:58 UTC
Created attachment 169469 [details] [review]
Bug 19316: Update items api definition

Signed-off-by: baptiste <baptiste.bayche@inlibro.com>
Comment 28 Matt Blenkinsop 2024-07-24 10:20:01 UTC
Created attachment 169470 [details] [review]
Bug 19316: Add CSRF protection

Signed-off-by: baptiste <baptiste.bayche@inlibro.com>
Comment 29 Matt Blenkinsop 2024-07-24 10:22:01 UTC
(In reply to Kyle M Hall from comment #23)
> I'm getting some odd behavior:
> 
> t/db_dependent/api/v1/items/bundled_items.t (Wstat: 512 (exited 2) Tests: 1
> Failed: 1)
>   Failed test:  1
>   Non-zero exit status: 2
>   Parse errors: Bad plan.  You planned 2 tests but ran 1.
> 
> ***************************************************************************
> DBIx::Class::Relationship::ManyToMany::many_to_many():
> ***************************************************************************
> The many-to-many relationship 'bundle_items' is trying to create a utility
> method
> called set_bundle_items.
> This will completely overwrite one such already existing method on class
> Koha::Schema::Result::Item.
> 
> You almost certainly want to rename your method or the many-to-many
> relationship, as the functionality of the original method will not be
> accessible anymore.
> 
> To disable this warning set to a true value the environment variable
> DBIC_OVERWRITE_HELPER_METHODS_OK

I can't recreate this, nor can I find a method anywhere called set_bundle_items
I installed the patch and ran a reset_all and the test passes fine - is there some additional config you have that I need to check with?

I've rebased one of the commits to fix some tidying issues on the QA tests and add Baptiste's sign off line to all the commits rather than just the final one
Comment 30 Kyle M Hall 2024-07-24 15:00:32 UTC
> I can't recreate this, nor can I find a method anywhere called
> set_bundle_items
> I installed the patch and ran a reset_all and the test passes fine - is
> there some additional config you have that I need to check with?

It could be something about my setup. Let's go back to SO and see if another QA'er can have a go at it!
Comment 31 Nick Clemens (kidclamp) 2024-07-24 15:57:55 UTC
Created attachment 169503 [details] [review]
Bug 19316: Add sort1 and sort2 to items and deleteditems tables

This patch adds to new fields to the items table to allow input for additional data into item records to help with sorting

Sponsored-by: PTFS Europe
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Comment 32 Nick Clemens (kidclamp) 2024-07-24 15:57:58 UTC
Created attachment 169504 [details] [review]
Bug 19316: Add DBIC files

Signed-off-by: baptiste <baptiste.bayche@inlibro.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Comment 33 Nick Clemens (kidclamp) 2024-07-24 15:58:00 UTC
Created attachment 169505 [details] [review]
Bug 19316: Add sort1 and sort2 fields to interface and allow editing

This patch adds the new sort1 and sort2 fields to the interface so that they can be viewed and edited

Test plan:
1) Apply patch and reset_all
2) In the catalog, search for a record with at least one item
3) In the Holdings table, click on the barcode for an item. You should now be on /cgi-bin/koha/catalogue/moredetail.pl
4) Scroll down and observe that there is now a section called "Sort fields" with two fields, Sort 1 and Sort 2.
5) Enter some text in the textarea for Sort 1 (max 50 characters) and click Update
6) The page will refresh and your text should still be visible. Check that item in the database and the text should be stored in the sort1 column.
7) Repeat steps 5-6 for Sort 2.
8) Copy the barcode and navigate to Cataloging > Batch item modification
9) Enter the barcode in the textarea under "Or scan items one by one" (You can add multiple bar codes here if you wish)
10) Fill in the Required values (Home library, Current library, Koha item type)
11) In "Other attributes" there should be two fields, Sort 1 and Sort 2
12) Enter some text into both these fields
13) Click save and a job will be enqueued
14) Click "view detail of the enqueued job"
15) A message should be shown on completion that says "X item(s) modified (with Y field(s) modified)." X should match the number of barcodes that you entered into the batch mod and Y should equal X x 2
16) Check the item either in the interface or the database and the text you entered should now be stored and visible
17) Repeat steps 8 - 14 but this time only fill in Sort 1
18) This time X should equal the number of barcodes that you entered into the batch mod but Y should equal X as only one field was filled in.
19) Again, the text you entered should now be stored on the item record

Signed-off-by: baptiste <baptiste.bayche@inlibro.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Comment 34 Nick Clemens (kidclamp) 2024-07-24 15:58:02 UTC
Created attachment 169506 [details] [review]
Bug 19316: Update items api definition

Signed-off-by: baptiste <baptiste.bayche@inlibro.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Comment 35 Nick Clemens (kidclamp) 2024-07-24 15:58:05 UTC
Created attachment 169507 [details] [review]
Bug 19316: Add CSRF protection

Signed-off-by: baptiste <baptiste.bayche@inlibro.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Comment 36 Katrin Fischer 2024-08-19 16:25:24 UTC
Hi Nick,

some questions:

1) It looks like these fields for items are MARC independent. They appear in the forms independent of a mapping to 952. Is this correct?

2) How would you import or export this data?

3) What would happen if they were mapped to a MARC subfield? (not in test plan)