Bug 38382

Summary: Need a fresh connection when CSRF has expired for connexion daemon
Product: Koha Reporter: Nick Clemens (kidclamp) <nick>
Component: Command-line UtilitiesAssignee: Nick Clemens (kidclamp) <nick>
Status: Pushed to main --- QA Contact: Marcel de Rooy <m.de.rooy>
Severity: normal    
Priority: P5 - low CC: blawlor, david, dcook, m.de.rooy, robin
Version: Main   
Hardware: All   
OS: All   
Change sponsored?: --- Patch complexity: Trivial patch
Documentation contact: Documentation submission:
Text to go in the release notes:
This fixes the OCLC Connexion import daemon (misc/bin/connexion_import_daemon.pl) - the connection was failing after the CSRF token expired (after 8 hours). It now generates a new user agent when reauthenticating when the CSRF token for the session has expired. (This is related to the CSRF changes added in Koha 24.05 to improve form security.)
Version(s) released in:
25.05.00
Circulation function:
Bug Depends on: 36192    
Bug Blocks:    
Attachments: Bug 38382: Fresh connection when connexion CSRF token expires
Sample record for import
Bug 38382: Fresh connection when connexion CSRF token expires
Bug 38382: Fresh connection when connexion CSRF token expires
Bug 38382: Fresh connection when connexion CSRF token expires

Description Nick Clemens (kidclamp) 2024-11-06 20:28:45 UTC
The CSRF token generated for Koha has a default 8 hour expiration. The connexion daemon tries to reauthnticate when it expires, however, it uses the same user agent, which has the session in the cookie, and so the CSRF fails the check.
Comment 1 Nick Clemens (kidclamp) 2024-11-06 20:37:52 UTC
Created attachment 174096 [details] [review]
Bug 38382: Fresh connection when connexion CSRF token expires

This patch simply generates a new user agent when reauthenticating when the CSRF token
for the session has expired.

To test:
1 - Edit Koha/Token.pm, line 275
from:    { MaxAge => $params->{MaxAge} // ( CSRF_EXPIRY_HOURS * 3600 ) },
to:      { MaxAge => $params->{MaxAge} // ( CSRF_EXPIRY_HOURS ) },
2 - This will make your CSRF tokens expire after 8 seconds
3 - vim /etc/koha/sites/kohadev/connexion.cnf
4 - Set content:
    host:
    port: 8888
    koha:http://localhost:8081
    log:/var/log/koha/kohadev/connexion.log
    match:ISBN
    user:kohauser
    password:kohapass
    overlay_action:replace
    nomatch_action:create_new
    item_action:always_add
    import_mode:direct
    framework:BKS
    overlay_framework:
    debug:1
5 - Save the sample file from this bug into your kohaclone (or copy and paste into a file your koha test site can reach)
6 - On the command line:
    perl misc/bin/connexion_import_daemon.pl -c /etc/koha/sites/kohadev/connexion.cnf
7 - In another terminal:
    cat bug_33418.test  | nc -v localhost 8888
8 - Successful request
9 - Wait 10 seconds (more than 8 anyways)
10 - Repeat 7
11 - Unsuccessful request - Invalid CSRF token
12 - Ctrl+C to stop the connexion daemon
13 - Apply patch
14 - repeat 6
15 - repeat 7, success
16 - Wait more than 8 seconds
17 - repeat 7, success!
18 - repeat again until you are satisfied
19 - Sign off!
Comment 2 Nick Clemens (kidclamp) 2024-11-06 20:47:51 UTC
Created attachment 174097 [details]
Sample record for import
Comment 3 Brendan Lawlor 2024-11-06 21:45:55 UTC
Created attachment 174098 [details] [review]
Bug 38382: Fresh connection when connexion CSRF token expires

This patch simply generates a new user agent when reauthenticating when the CSRF token
for the session has expired.

To test:
1 - Edit Koha/Token.pm, line 275
from:    { MaxAge => $params->{MaxAge} // ( CSRF_EXPIRY_HOURS * 3600 ) },
to:      { MaxAge => $params->{MaxAge} // ( CSRF_EXPIRY_HOURS ) },
2 - This will make your CSRF tokens expire after 8 seconds
3 - vim /etc/koha/sites/kohadev/connexion.cnf
4 - Set content:
    host:
    port: 8888
    koha:http://localhost:8081
    log:/var/log/koha/kohadev/connexion.log
    match:ISBN
    user:kohauser
    password:kohapass
    overlay_action:replace
    nomatch_action:create_new
    item_action:always_add
    import_mode:direct
    framework:BKS
    overlay_framework:
    debug:1
5 - Save the sample file from this bug into your kohaclone (or copy and paste into a file your koha test site can reach)
6 - On the command line:
    perl misc/bin/connexion_import_daemon.pl -c /etc/koha/sites/kohadev/connexion.cnf
7 - In another terminal:
    cat bug_33418.test  | nc -v localhost 8888
8 - Successful request
9 - Wait 10 seconds (more than 8 anyways)
10 - Repeat 7
11 - Unsuccessful request - Invalid CSRF token
12 - Ctrl+C to stop the connexion daemon
13 - Apply patch
14 - repeat 6
15 - repeat 7, success
16 - Wait more than 8 seconds
17 - repeat 7, success!
18 - repeat again until you are satisfied
19 - Sign off!

Signed-off-by: Brendan Lawlor <blawlor@clamsnet.org>
Comment 4 Brendan Lawlor 2024-11-06 21:51:42 UTC
Test notes:

This patch works as described.
Updated Token.pm MaxAge

Before patch after 8 seconds resopnses would come back like
Wed Nov  6 21:39:09 2024: HTTP/1.1 403 Forbidden
Connection: close
Date: Wed, 06 Nov 2024 21:39:09 GMT
Server: Apache/2.4.62 (Debian)
Vary: User-Agent
Content-Type: text/plain
Client-Date: Wed, 06 Nov 2024 21:39:09 GMT
Client-Peer: 127.0.0.1:8081
Client-Response-Num: 1
Client-Transfer-Encoding: chunked

Wrong CSRF token

Wed Nov  6 21:39:09 2024: Response: Unsuccessful request

After patch response would continue to come back successful:
Wed Nov  6 21:44:16 2024: U8kohauserP8kohapass00575nam  22002055  450000100070000000500170000700800410002402000180006503500180008304000140010110000300011524500250014526000200017030000200019094200120021095201070022295500210032995600080035099900110035896475820240222153924.0981015s1948    enk           000   eng d  a9780679777427  aCAT1-0666609A  aNLScKOHA1 aYerby, Frank Garvin.923514aThe vixens. A novel.  aLondon, c1948.  a326 p. ; c8vo.  2ddccBK  0010406_708REF960aMPLbMPLcGENd2014-09-04p39999000001051r2014-09-04 00:00:00w2014-09-04yBK  aVts.178.i.2bVts  aNLS  c24d24

Wed Nov  6 21:44:16 2024: Response: Success.  Batch number 9 - biblio record number 442 overlaid by import.

http://kohadev-intra.myDNSname.org:8081/cgi-bin/koha/catalogue/detail.pl?biblionumber=442
Comment 5 Brendan Lawlor 2024-11-06 21:57:24 UTC
When starting and stopping the daemon sometimes I'd get a port in use error and this command came in handy:
sudo fuser -k 8888/tcp

Also When I followed the test plan for Bug 37543 I saw the success message showing HASH(#) instead of the bib number as described in bug 38230

But when I followed this test plan the response correctly displayed the bib number.

So I guess that issue seems to be replated to config since I was sending the same requests when I was testing.
Comment 6 Marcel de Rooy 2024-11-22 08:07:24 UTC
 WARN   misc/bin/connexion_import_daemon.pl
   WARN   tidiness
                The file is less tidy than before (bad/messy lines before: 0, now: 3)
Comment 7 Nick Clemens (kidclamp) 2024-12-05 16:52:53 UTC
Created attachment 175250 [details] [review]
Bug 38382: Fresh connection when connexion CSRF token expires

This patch simply generates a new user agent when reauthenticating when the CSRF token
for the session has expired.

To test:
1 - Edit Koha/Token.pm, line 275
from:    { MaxAge => $params->{MaxAge} // ( CSRF_EXPIRY_HOURS * 3600 ) },
to:      { MaxAge => $params->{MaxAge} // ( CSRF_EXPIRY_HOURS ) },
2 - This will make your CSRF tokens expire after 8 seconds
3 - vim /etc/koha/sites/kohadev/connexion.cnf
4 - Set content:
    host:
    port: 8888
    koha:http://localhost:8081
    log:/var/log/koha/kohadev/connexion.log
    match:ISBN
    user:kohauser
    password:kohapass
    overlay_action:replace
    nomatch_action:create_new
    item_action:always_add
    import_mode:direct
    framework:BKS
    overlay_framework:
    debug:1
5 - Save the sample file from this bug into your kohaclone (or copy and paste into a file your koha test site can reach)
6 - On the command line:
    perl misc/bin/connexion_import_daemon.pl -c /etc/koha/sites/kohadev/connexion.cnf
7 - In another terminal:
    cat bug_33418.test  | nc -v localhost 8888
8 - Successful request
9 - Wait 10 seconds (more than 8 anyways)
10 - Repeat 7
11 - Unsuccessful request - Invalid CSRF token
12 - Ctrl+C to stop the connexion daemon
13 - Apply patch
14 - repeat 6
15 - repeat 7, success
16 - Wait more than 8 seconds
17 - repeat 7, success!
18 - repeat again until you are satisfied
19 - Sign off!

WNC amended and tidied

Signed-off-by: Brendan Lawlor <blawlor@clamsnet.org>
Comment 8 Marcel de Rooy 2024-12-13 10:42:13 UTC
Created attachment 175444 [details] [review]
Bug 38382: Fresh connection when connexion CSRF token expires

This patch simply generates a new user agent when reauthenticating when the CSRF token
for the session has expired.

To test:
1 - Edit Koha/Token.pm, line 275
from:    { MaxAge => $params->{MaxAge} // ( CSRF_EXPIRY_HOURS * 3600 ) },
to:      { MaxAge => $params->{MaxAge} // ( CSRF_EXPIRY_HOURS ) },
2 - This will make your CSRF tokens expire after 8 seconds
3 - vim /etc/koha/sites/kohadev/connexion.cnf
4 - Set content:
    host:
    port: 8888
    koha:http://localhost:8081
    log:/var/log/koha/kohadev/connexion.log
    match:ISBN
    user:kohauser
    password:kohapass
    overlay_action:replace
    nomatch_action:create_new
    item_action:always_add
    import_mode:direct
    framework:BKS
    overlay_framework:
    debug:1
5 - Save the sample file from this bug into your kohaclone (or copy and paste into a file your koha test site can reach)
6 - On the command line:
    perl misc/bin/connexion_import_daemon.pl -c /etc/koha/sites/kohadev/connexion.cnf
7 - In another terminal:
    cat bug_33418.test  | nc -v localhost 8888
8 - Successful request
9 - Wait 10 seconds (more than 8 anyways)
10 - Repeat 7
11 - Unsuccessful request - Invalid CSRF token
12 - Ctrl+C to stop the connexion daemon
13 - Apply patch
14 - repeat 6
15 - repeat 7, success
16 - Wait more than 8 seconds
17 - repeat 7, success!
18 - repeat again until you are satisfied
19 - Sign off!

WNC amended and tidied

Signed-off-by: Brendan Lawlor <blawlor@clamsnet.org>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Comment 9 Katrin Fischer 2024-12-23 08:38:08 UTC
Pushed for 25.05!

Well done everyone, thank you!