Bug 37582 - SIP2 responses can contain newlines when a patron has multiple debarments
Summary: SIP2 responses can contain newlines when a patron has multiple debarments
Status: Pushed to main
Alias: None
Product: Koha
Classification: Unclassified
Component: SIP2 (show other bugs)
Version: Main
Hardware: All All
: P5 - low normal
Assignee: Kyle M Hall (khall)
QA Contact: Martin Renvoize (ashimema)
URL:
Keywords: rel_24_05_candidate
Depends on:
Blocks:
 
Reported: 2024-08-06 17:17 UTC by Nick Clemens (kidclamp)
Modified: 2024-10-30 15:18 UTC (History)
4 users (show)

See Also:
Change sponsored?: ---
Patch complexity: ---
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:
24.11.00
Circulation function:


Attachments
Bug 37582: Add ability to for SIP to convert control and separator characters to spaces (9.42 KB, patch)
2024-09-17 18:46 UTC, Kyle M Hall (khall)
Details | Diff | Splinter Review
Bug 37582: Add ability to for SIP to convert control and separator characters to spaces (9.52 KB, patch)
2024-09-17 18:53 UTC, Kyle M Hall (khall)
Details | Diff | Splinter Review
Bug 37582: Tidy write_msg (1.63 KB, patch)
2024-09-17 18:53 UTC, Kyle M Hall (khall)
Details | Diff | Splinter Review
Bug 37582: Add ability to for SIP to convert control and separator characters to spaces (9.55 KB, patch)
2024-10-21 20:11 UTC, Brendan Lawlor
Details | Diff | Splinter Review
Bug 37582: Tidy write_msg (1.66 KB, patch)
2024-10-21 20:11 UTC, Brendan Lawlor
Details | Diff | Splinter Review
Bug 37582: Tidy write_msg (1.72 KB, patch)
2024-10-24 20:06 UTC, ByWater Sandboxes
Details | Diff | Splinter Review
Bug 37582: Add ability to for SIP to convert control and separator characters to spaces (9.62 KB, patch)
2024-10-28 10:39 UTC, Martin Renvoize (ashimema)
Details | Diff | Splinter Review
Bug 37582: Tidy write_msg (1.79 KB, patch)
2024-10-28 10:39 UTC, Martin Renvoize (ashimema)
Details | Diff | Splinter Review
Bug 37582: Add unit test [WIP] (2.10 KB, patch)
2024-10-30 11:55 UTC, Kyle M Hall (khall)
Details | Diff | Splinter Review
Bug 37582: Add unit test (1.68 KB, patch)
2024-10-30 12:25 UTC, Kyle M Hall (khall)
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Nick Clemens (kidclamp) 2024-08-06 17:17:00 UTC
This came up with a SIP vendor recently - when reading responses they were only getting part of a patron's information. It seemed to have been caused by patrons with multiple restrictions and a new line in the response

The simple fix was:
<screen_msg_regex find="\n" replace=" -- " />

We should possibly add this as an option rather than relying on regex
Comment 1 Kyle M Hall (khall) 2024-09-17 18:46:48 UTC
Created attachment 171642 [details] [review]
Bug 37582: Add ability to for SIP to convert control and separator characters to spaces

This came up with a SIP vendor recently - when reading responses they were only getting part of a patron's information.
It seemed to have been caused by patrons with multiple restrictions and a new line in the response

Test Plan:
1) Add multiple debarments to a patron
2) Run a sip patron information response, note the presence of newlines
3) Apply this patch
4) Add the new option convert_control_characters to your SIP account,
   set the value to " -- "
5) Restart all the things!
7) Run a sip patron information response, note the newlines have been replaced!
Comment 2 Kyle M Hall (khall) 2024-09-17 18:53:20 UTC
Created attachment 171643 [details] [review]
Bug 37582: Add ability to for SIP to convert control and separator characters to spaces

This came up with a SIP vendor recently - when reading responses they were only getting part of a patron's information.
It seemed to have been caused by patrons with multiple restrictions and a new line in the response

This patch also does minor refactoring to write_msg to pass the server variable that contains
all the previously passed parameters, and removes the never used "file" parameter.

Test Plan:
1) Add multiple debarments to a patron
2) Run a sip patron information response, note the presence of newlines
3) Apply this patch
4) Add the new option convert_control_characters to your SIP account,
   set the value to " -- "
5) Restart all the things!
7) Run a sip patron information response, note the newlines have been replaced!
Comment 3 Kyle M Hall (khall) 2024-09-17 18:53:26 UTC
Created attachment 171644 [details] [review]
Bug 37582: Tidy write_msg
Comment 4 Brendan Lawlor 2024-10-17 20:56:23 UTC
I got pretty close to being able to test this, but I'm missing something.

In /etc/koha/SIPconfig.xml I added the new option like this:
<login id="term1"  password="term1" delimiter="|" error-detect="enabled" institu    tion="CPL" encoding="ascii" checked_in_ok="1" convert_nonprinting_characters=" -- "/>

Then I restart_all and login to the SIP server and send a patron info request, but I still got a response with a newline after the first -- debarred message

kohadev-koha@kohadevbox:koha(main)$ telnet localhost 6001
Trying ::1...
Connection failed: Connection refused
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
9300CNterm1|COterm1|CPCPL|
941
6300020060329    201700Y         AOCPL|AAbob|AD1234|
64YYYY          00020241017    205014000000010001000000000000AOCPL|AA23529000695412|AEBob Singleton|BLY|CQY|BHUSD|BV0|CC5|BD4554 Library Rd. Centerville, CA 44262|BF(212) 555-1212|PB19830415|PCPT|PIN|AFGreetings from Koha.  -- debarred
debarred again|
Comment 5 Kyle M Hall (khall) 2024-10-21 12:56:12 UTC
(In reply to Brendan Lawlor from comment #4)
> In /etc/koha/SIPconfig.xml I added the new option like this:
> <login id="term1"  password="term1" delimiter="|" error-detect="enabled"
> institu    tion="CPL" encoding="ascii" checked_in_ok="1"
> convert_nonprinting_characters=" -- "/>

I think you may be updating the wrong file. For koha-testing-docker the file to update would be /etc/koha/sites/kohadev/SIPconfig.xml 🤞
Comment 6 Brendan Lawlor 2024-10-21 20:11:11 UTC
Created attachment 173081 [details] [review]
Bug 37582: Add ability to for SIP to convert control and separator characters to spaces

This came up with a SIP vendor recently - when reading responses they were only getting part of a patron's information.
It seemed to have been caused by patrons with multiple restrictions and a new line in the response

This patch also does minor refactoring to write_msg to pass the server variable that contains
all the previously passed parameters, and removes the never used "file" parameter.

Test Plan:
1) Add multiple debarments to a patron
2) Run a sip patron information response, note the presence of newlines
3) Apply this patch
4) Add the new option convert_control_characters to your SIP account,
   set the value to " -- "
5) Restart all the things!
7) Run a sip patron information response, note the newlines have been replaced!

Signed-off-by: Brendan Lawlor <blawlor@clamsnet.org>
Comment 7 Brendan Lawlor 2024-10-21 20:11:13 UTC
Created attachment 173082 [details] [review]
Bug 37582: Tidy write_msg

Signed-off-by: Brendan Lawlor <blawlor@clamsnet.org>
Comment 8 Brendan Lawlor 2024-10-21 20:15:13 UTC
This works, with one small correction to the test plan.

In step #4 the new option should be convert_nonprinting_characters=" -- "
Comment 9 Kyle M Hall (khall) 2024-10-22 13:00:20 UTC
(In reply to Brendan Lawlor from comment #8)
> This works, with one small correction to the test plan.
> 
> In step #4 the new option should be convert_nonprinting_characters=" -- "

Ack! Thanks for catching that!
Comment 10 ByWater Sandboxes 2024-10-24 20:06:57 UTC
Created attachment 173306 [details] [review]
Bug 37582: Tidy write_msg

Signed-off-by: Brendan Lawlor <blawlor@clamsnet.org>
Signed-off-by: Kristi Krueger <kkrueger@cuyahogalibrary.org>
Comment 11 Martin Renvoize (ashimema) 2024-10-28 10:39:49 UTC
Created attachment 173502 [details] [review]
Bug 37582: Add ability to for SIP to convert control and separator characters to spaces

This came up with a SIP vendor recently - when reading responses they were only getting part of a patron's information.
It seemed to have been caused by patrons with multiple restrictions and a new line in the response

This patch also does minor refactoring to write_msg to pass the server variable that contains
all the previously passed parameters, and removes the never used "file" parameter.

Test Plan:
1) Add multiple debarments to a patron
2) Run a sip patron information response, note the presence of newlines
3) Apply this patch
4) Add the new option convert_control_characters to your SIP account,
   set the value to " -- "
5) Restart all the things!
7) Run a sip patron information response, note the newlines have been replaced!

Signed-off-by: Brendan Lawlor <blawlor@clamsnet.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 12 Martin Renvoize (ashimema) 2024-10-28 10:39:52 UTC
Created attachment 173503 [details] [review]
Bug 37582: Tidy write_msg

Signed-off-by: Brendan Lawlor <blawlor@clamsnet.org>
Signed-off-by: Kristi Krueger <kkrueger@cuyahogalibrary.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 13 Katrin Fischer 2024-10-29 07:26:24 UTC
Could we have a unit test for this new option and the changes please?
Comment 14 Katrin Fischer 2024-10-29 07:54:10 UTC
Pushed for 24.11!

Well done everyone, thank you!
Comment 15 Kyle M Hall (khall) 2024-10-30 11:55:10 UTC
Created attachment 173712 [details] [review]
Bug 37582: Add unit test [WIP]
Comment 16 Kyle M Hall (khall) 2024-10-30 12:25:11 UTC
Created attachment 173715 [details] [review]
Bug 37582: Add unit test
Comment 17 Kyle M Hall (khall) 2024-10-30 12:25:42 UTC
Unit test ready to go!
Comment 18 Katrin Fischer 2024-10-30 13:02:11 UTC
Thanks, follow-up unit tests grabbed for main :)