Bug 41182 - SUSHI harvest should send a full date
Summary: SUSHI harvest should send a full date
Status: Failed QA
Alias: None
Product: Koha
Classification: Unclassified
Component: ERM (show other bugs)
Version: Main
Hardware: All All
: P5 - low normal
Assignee: Nick Clemens (kidclamp)
QA Contact: Pedro Amorim
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2025-11-04 17:57 UTC by Nick Clemens (kidclamp)
Modified: 2026-01-26 20:08 UTC (History)
7 users (show)

See Also:
GIT URL:
Initiative type: ---
Sponsorship status: ---
Comma delimited list of Sponsors:
Crowdfunding goal: 0
Patch complexity: ---
Documentation contact:
Documentation submission:
Text to go in the release notes:
This updates how the date is sent for SUSHI harvest requests, as it was failing for Proquest counter requests. The date is now sent in full as 10 digits (YYYY-MM-DD), instead of 7 digits (YYYY-MM). Example: - Previously sent as: https://sushi.proquest.com/counter/r5/reports/pr?begin_date=2024-11&end_date=2025-10&customer_id={ID}&api_key={KEY} - Now sent as: https://sushi.proquest.com/counter/r5/reports/pr?begin_date=2024-11-01&end_date=2025-10-31&customer_id={ID}&api_key={KEY}
Version(s) released in:
Circulation function:


Attachments
Bug 41182: Pass 10 digit date to SUSHI (1.19 KB, patch)
2025-11-04 18:02 UTC, Nick Clemens (kidclamp)
Details | Diff | Splinter Review
Bug 41182: Pass 10 digit date to SUSHI (1.24 KB, patch)
2025-12-24 17:42 UTC, David Nind
Details | Diff | Splinter Review
Bug 41182: Pass 10 digit date to SUSHI (2.03 KB, patch)
2026-01-26 16:15 UTC, Nick Clemens (kidclamp)
Details | Diff | Splinter Review
Bug 41182: DO NOT PUSH - Schema updates (1.54 KB, patch)
2026-01-26 16:15 UTC, Nick Clemens (kidclamp)
Details | Diff | Splinter Review
Bug 41182: Database changes (2.70 KB, patch)
2026-01-26 16:15 UTC, Nick Clemens (kidclamp)
Details | Diff | Splinter Review
Bug 41182: Adjust interface for new field (3.39 KB, patch)
2026-01-26 16:15 UTC, Nick Clemens (kidclamp)
Details | Diff | Splinter Review
Bug 41182: Unit tests (3.45 KB, patch)
2026-01-26 16:15 UTC, Nick Clemens (kidclamp)
Details | Diff | Splinter Review
Bug 41182: Allow choosing format of date passed to SUSHI (2.39 KB, patch)
2026-01-26 17:01 UTC, Nick Clemens (kidclamp)
Details | Diff | Splinter Review
Bug 41182: Database changes (2.82 KB, patch)
2026-01-26 17:02 UTC, Nick Clemens (kidclamp)
Details | Diff | Splinter Review
Bug 41182: Adjust interface for new field (3.42 KB, patch)
2026-01-26 17:02 UTC, Nick Clemens (kidclamp)
Details | Diff | Splinter Review
Bug 41182: Unit tests (3.48 KB, patch)
2026-01-26 17:02 UTC, Nick Clemens (kidclamp)
Details | Diff | Splinter Review
Bug 41182: DO NOT PUSH - Schema updates (1.67 KB, patch)
2026-01-26 17:02 UTC, Nick Clemens (kidclamp)
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Nick Clemens (kidclamp) 2025-11-04 17:57:58 UTC
A library attempting to get counter reports from Proquest found they were failing.

In testing, I was able to narrow the problem down to the date range.

We send:
https://sushi.proquest.com/counter/r5/reports/pr?begin_date=2024-11&end_date=2025-10&customer_id={ID}&api_key={KEY}

They expect:
https://sushi.proquest.com/counter/r5/reports/pr?begin_date=2024-11-01&end_date=2025-10-31&customer_id={ID}&api_key={KEY}

From what I can tell in the standard:
https://www.niso.org/sites/default/files/schema-files/sushi1_7_ReportRequest.png
Date is expected as yyyy-mm-dd

We should send the full date and not truncate
Comment 1 Nick Clemens (kidclamp) 2025-11-04 18:02:05 UTC
Created attachment 189044 [details] [review]
Bug 41182: Pass 10 digit date to SUSHI
Comment 2 David Nind 2025-12-24 17:42:23 UTC
Created attachment 190713 [details] [review]
Bug 41182: Pass 10 digit date to SUSHI

Signed-off-by: David Nind <david@davidnind.com>
Comment 3 David Nind 2025-12-24 18:06:59 UTC
I've signed off, even though I don't have a way to test.

Have updated the assignee.
Comment 4 Nick Clemens (kidclamp) 2026-01-26 16:15:46 UTC
Created attachment 192041 [details] [review]
Bug 41182: Pass 10 digit date to SUSHI

These patches add a new dateformat field to erm_usage_data_providers
to allow for choosing the format sent to sushi

To test:
1 - Enable ERMModule
2 - Add a new Data providers
3 - Click 'Add manually' and add a provider without setting the 'Date format' field
4 - Confirm the provider is saved with dateformat 7 digit
5 - Edit and set the provider date format to 10 digit
6 - Confirm it saves correctly
7 - Edit and set to 7 and confirm saved
8 - Edit and clear the field, it saves as 7
9 - Run unit tests
Comment 5 Nick Clemens (kidclamp) 2026-01-26 16:15:49 UTC
Created attachment 192042 [details] [review]
Bug 41182: DO NOT PUSH - Schema updates
Comment 6 Nick Clemens (kidclamp) 2026-01-26 16:15:51 UTC
Created attachment 192043 [details] [review]
Bug 41182: Database changes
Comment 7 Nick Clemens (kidclamp) 2026-01-26 16:15:53 UTC
Created attachment 192044 [details] [review]
Bug 41182: Adjust interface for new field
Comment 8 Nick Clemens (kidclamp) 2026-01-26 16:15:56 UTC
Created attachment 192045 [details] [review]
Bug 41182: Unit tests
Comment 9 Nick Clemens (kidclamp) 2026-01-26 16:16:23 UTC
Updated to allow for choice of date format - back to NSO
Comment 10 Nick Clemens (kidclamp) 2026-01-26 17:01:59 UTC
Created attachment 192046 [details] [review]
Bug 41182: Allow choosing format of date passed to SUSHI

These patches add a new dateformat field to erm_usage_data_providers
to allow for choosing the format sent to sushi

The formats supported are:
yyyy-mm
yyyy-mm-dd

yyyy-mm is the current assumption and remains the default

To test:
 0 - Apply patches, updatedatabase, yarn build, restart all
 1 - Enable ERMModule
 2 - Add a new Data providers
 3 - Click 'Add manually' and add a provider without setting the 'Date format' field
 4 - Confirm the provider is saved with dateformat 'YYYY-MM'
 5 - Edit and set the provider date format to 'YYYY-MM-DD'
 6 - Confirm it saves correctly
 7 - Edit and set to 'YYYY-MM' and confirm saved
 8 - Edit and clear the field, it remains as 'YYYY-MM'
 9 - Edit and save as 'YYYY-MM-DD'
10 - Edit and clear the field and save
11 - Confirm it remains 'YYYY-MM-DD'
12 - Run unit tests
Comment 11 Nick Clemens (kidclamp) 2026-01-26 17:02:01 UTC
Created attachment 192047 [details] [review]
Bug 41182: Database changes
Comment 12 Nick Clemens (kidclamp) 2026-01-26 17:02:04 UTC
Created attachment 192048 [details] [review]
Bug 41182: Adjust interface for new field
Comment 13 Nick Clemens (kidclamp) 2026-01-26 17:02:07 UTC
Created attachment 192049 [details] [review]
Bug 41182: Unit tests
Comment 14 Nick Clemens (kidclamp) 2026-01-26 17:02:09 UTC
Created attachment 192050 [details] [review]
Bug 41182: DO NOT PUSH - Schema updates
Comment 15 David Nind 2026-01-26 20:08:37 UTC
For me, for step 8, the date remains as YYYY-MM-DD (instead of YYYY-MM as stated in the test plan).

Testing notes (using KTD):

1. To run tests: prove t/db_dependent/Koha/ERM/EUsage/UsageDataProvider.t