The CSRF token generated for Koha has a default 8 hour expiration. The connexion daemon tries to reauthnticate when it expires, however, it uses the same user agent, which has the session in the cookie, and so the CSRF fails the check.
Created attachment 174096 [details] [review] Bug 38382: Fresh connection when connexion CSRF token expires This patch simply generates a new user agent when reauthenticating when the CSRF token for the session has expired. To test: 1 - Edit Koha/Token.pm, line 275 from: { MaxAge => $params->{MaxAge} // ( CSRF_EXPIRY_HOURS * 3600 ) }, to: { MaxAge => $params->{MaxAge} // ( CSRF_EXPIRY_HOURS ) }, 2 - This will make your CSRF tokens expire after 8 seconds 3 - vim /etc/koha/sites/kohadev/connexion.cnf 4 - Set content: host: port: 8888 koha:http://localhost:8081 log:/var/log/koha/kohadev/connexion.log match:ISBN user:kohauser password:kohapass overlay_action:replace nomatch_action:create_new item_action:always_add import_mode:direct framework:BKS overlay_framework: debug:1 5 - Save the sample file from this bug into your kohaclone (or copy and paste into a file your koha test site can reach) 6 - On the command line: perl misc/bin/connexion_import_daemon.pl -c /etc/koha/sites/kohadev/connexion.cnf 7 - In another terminal: cat bug_33418.test | nc -v localhost 8888 8 - Successful request 9 - Wait 10 seconds (more than 8 anyways) 10 - Repeat 7 11 - Unsuccessful request - Invalid CSRF token 12 - Ctrl+C to stop the connexion daemon 13 - Apply patch 14 - repeat 6 15 - repeat 7, success 16 - Wait more than 8 seconds 17 - repeat 7, success! 18 - repeat again until you are satisfied 19 - Sign off!
Created attachment 174097 [details] Sample record for import
Created attachment 174098 [details] [review] Bug 38382: Fresh connection when connexion CSRF token expires This patch simply generates a new user agent when reauthenticating when the CSRF token for the session has expired. To test: 1 - Edit Koha/Token.pm, line 275 from: { MaxAge => $params->{MaxAge} // ( CSRF_EXPIRY_HOURS * 3600 ) }, to: { MaxAge => $params->{MaxAge} // ( CSRF_EXPIRY_HOURS ) }, 2 - This will make your CSRF tokens expire after 8 seconds 3 - vim /etc/koha/sites/kohadev/connexion.cnf 4 - Set content: host: port: 8888 koha:http://localhost:8081 log:/var/log/koha/kohadev/connexion.log match:ISBN user:kohauser password:kohapass overlay_action:replace nomatch_action:create_new item_action:always_add import_mode:direct framework:BKS overlay_framework: debug:1 5 - Save the sample file from this bug into your kohaclone (or copy and paste into a file your koha test site can reach) 6 - On the command line: perl misc/bin/connexion_import_daemon.pl -c /etc/koha/sites/kohadev/connexion.cnf 7 - In another terminal: cat bug_33418.test | nc -v localhost 8888 8 - Successful request 9 - Wait 10 seconds (more than 8 anyways) 10 - Repeat 7 11 - Unsuccessful request - Invalid CSRF token 12 - Ctrl+C to stop the connexion daemon 13 - Apply patch 14 - repeat 6 15 - repeat 7, success 16 - Wait more than 8 seconds 17 - repeat 7, success! 18 - repeat again until you are satisfied 19 - Sign off! Signed-off-by: Brendan Lawlor <blawlor@clamsnet.org>
Test notes: This patch works as described. Updated Token.pm MaxAge Before patch after 8 seconds resopnses would come back like Wed Nov 6 21:39:09 2024: HTTP/1.1 403 Forbidden Connection: close Date: Wed, 06 Nov 2024 21:39:09 GMT Server: Apache/2.4.62 (Debian) Vary: User-Agent Content-Type: text/plain Client-Date: Wed, 06 Nov 2024 21:39:09 GMT Client-Peer: 127.0.0.1:8081 Client-Response-Num: 1 Client-Transfer-Encoding: chunked Wrong CSRF token Wed Nov 6 21:39:09 2024: Response: Unsuccessful request After patch response would continue to come back successful: Wed Nov 6 21:44:16 2024: U8kohauserP8kohapass00575nam 22002055 450000100070000000500170000700800410002402000180006503500180008304000140010110000300011524500250014526000200017030000200019094200120021095201070022295500210032995600080035099900110035896475820240222153924.0981015s1948 enk 000 eng d a9780679777427 aCAT1-0666609A aNLScKOHA1 aYerby, Frank Garvin.923514aThe vixens. A novel. aLondon, c1948. a326 p. ; c8vo. 2ddccBK 0010406_708REF960aMPLbMPLcGENd2014-09-04p39999000001051r2014-09-04 00:00:00w2014-09-04yBK aVts.178.i.2bVts aNLS c24d24 Wed Nov 6 21:44:16 2024: Response: Success. Batch number 9 - biblio record number 442 overlaid by import. http://kohadev-intra.myDNSname.org:8081/cgi-bin/koha/catalogue/detail.pl?biblionumber=442
When starting and stopping the daemon sometimes I'd get a port in use error and this command came in handy: sudo fuser -k 8888/tcp Also When I followed the test plan for Bug 37543 I saw the success message showing HASH(#) instead of the bib number as described in bug 38230 But when I followed this test plan the response correctly displayed the bib number. So I guess that issue seems to be replated to config since I was sending the same requests when I was testing.
WARN misc/bin/connexion_import_daemon.pl WARN tidiness The file is less tidy than before (bad/messy lines before: 0, now: 3)
Created attachment 175250 [details] [review] Bug 38382: Fresh connection when connexion CSRF token expires This patch simply generates a new user agent when reauthenticating when the CSRF token for the session has expired. To test: 1 - Edit Koha/Token.pm, line 275 from: { MaxAge => $params->{MaxAge} // ( CSRF_EXPIRY_HOURS * 3600 ) }, to: { MaxAge => $params->{MaxAge} // ( CSRF_EXPIRY_HOURS ) }, 2 - This will make your CSRF tokens expire after 8 seconds 3 - vim /etc/koha/sites/kohadev/connexion.cnf 4 - Set content: host: port: 8888 koha:http://localhost:8081 log:/var/log/koha/kohadev/connexion.log match:ISBN user:kohauser password:kohapass overlay_action:replace nomatch_action:create_new item_action:always_add import_mode:direct framework:BKS overlay_framework: debug:1 5 - Save the sample file from this bug into your kohaclone (or copy and paste into a file your koha test site can reach) 6 - On the command line: perl misc/bin/connexion_import_daemon.pl -c /etc/koha/sites/kohadev/connexion.cnf 7 - In another terminal: cat bug_33418.test | nc -v localhost 8888 8 - Successful request 9 - Wait 10 seconds (more than 8 anyways) 10 - Repeat 7 11 - Unsuccessful request - Invalid CSRF token 12 - Ctrl+C to stop the connexion daemon 13 - Apply patch 14 - repeat 6 15 - repeat 7, success 16 - Wait more than 8 seconds 17 - repeat 7, success! 18 - repeat again until you are satisfied 19 - Sign off! WNC amended and tidied Signed-off-by: Brendan Lawlor <blawlor@clamsnet.org>
Created attachment 175444 [details] [review] Bug 38382: Fresh connection when connexion CSRF token expires This patch simply generates a new user agent when reauthenticating when the CSRF token for the session has expired. To test: 1 - Edit Koha/Token.pm, line 275 from: { MaxAge => $params->{MaxAge} // ( CSRF_EXPIRY_HOURS * 3600 ) }, to: { MaxAge => $params->{MaxAge} // ( CSRF_EXPIRY_HOURS ) }, 2 - This will make your CSRF tokens expire after 8 seconds 3 - vim /etc/koha/sites/kohadev/connexion.cnf 4 - Set content: host: port: 8888 koha:http://localhost:8081 log:/var/log/koha/kohadev/connexion.log match:ISBN user:kohauser password:kohapass overlay_action:replace nomatch_action:create_new item_action:always_add import_mode:direct framework:BKS overlay_framework: debug:1 5 - Save the sample file from this bug into your kohaclone (or copy and paste into a file your koha test site can reach) 6 - On the command line: perl misc/bin/connexion_import_daemon.pl -c /etc/koha/sites/kohadev/connexion.cnf 7 - In another terminal: cat bug_33418.test | nc -v localhost 8888 8 - Successful request 9 - Wait 10 seconds (more than 8 anyways) 10 - Repeat 7 11 - Unsuccessful request - Invalid CSRF token 12 - Ctrl+C to stop the connexion daemon 13 - Apply patch 14 - repeat 6 15 - repeat 7, success 16 - Wait more than 8 seconds 17 - repeat 7, success! 18 - repeat again until you are satisfied 19 - Sign off! WNC amended and tidied Signed-off-by: Brendan Lawlor <blawlor@clamsnet.org> Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Pushed for 25.05! Well done everyone, thank you!
Nice work everyone! Pushed to 24.11.x for 24.11.01
Hi, 1) Can you please give us a more detailed test plan? 2) Are we supposed to make the /etc/koha/sites/kohadev/connexion.cnf file? 3) We noticed that there are some strange characters in the sample record for import file ( https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=174097 ), which is giving us the following output in step 6: kohadev-koha@kohadevbox:koha((v24.05.06))$ perl misc/bin/connexion_import_daemon.pl -c /etc/koha/sites/kohadev/connexion.cnf no mapping found for [0x90] at position 0 in ��96475 g0=ASCII_DEFAULT g1=EXTENDED_LATIN at /usr/share/perl5/MARC/Charset.pm line 308. no mapping found for [0x90] at position 2 in 8␞2024022215392 g0=ASCII_DEFAULT g1=EXTENDED_LATIN at /usr/share/perl5/MARC/Charset.pm line 308. no mapping found for [0x90] at position 4 in 4.0␞981015s1948 enk 000 g0=ASCII_DEFAULT g1=EXTENDED_LATIN at /usr/share/perl5/MARC/Charset.pm line 308. 4) For step 8 we get the following, is this to be expected? kohadev-koha@kohadevbox:koha((v24.05.06))$ cat bug_33418.test | nc -v localhost 8888 nc: connect to localhost (::1) port 8888 (tcp) failed: Connection refused Connection to localhost (127.0.0.1) 8888 port [tcp/*] succeeded! Unsuccessful requestkohadev-koha@kohadevbox:koha((v24.05.06))$
(In reply to Alex Buckley from comment #11) > Hi, > > 1) Can you please give us a more detailed test plan? Which parts are unclear? I should have mentioned creating a user with kohauser/kohapass as the credentials, or use the credentials for an existing account with the correct permissions > 2) Are we supposed to make the /etc/koha/sites/kohadev/connexion.cnf file? Yes, create this file if it doesn't exist > 3) We noticed that there are some strange characters in the sample record > for import file ( > https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=174097 ), which > is giving us the following output in step 6: > > kohadev-koha@kohadevbox:koha((v24.05.06))$ perl > misc/bin/connexion_import_daemon.pl -c /etc/koha/sites/kohadev/connexion.cnf > no mapping found for [0x90] at position 0 in ��96475 g0=ASCII_DEFAULT > g1=EXTENDED_LATIN at /usr/share/perl5/MARC/Charset.pm line 308. > no mapping found for [0x90] at position 2 in 8␞2024022215392 > g0=ASCII_DEFAULT g1=EXTENDED_LATIN at /usr/share/perl5/MARC/Charset.pm line > 308. > no mapping found for [0x90] at position 4 in 4.0␞981015s1948 enk > 000 g0=ASCII_DEFAULT g1=EXTENDED_LATIN at > /usr/share/perl5/MARC/Charset.pm line 308. I think this may be from copying/pasting or the way it was downloaded - you can simply export a single record from your koha in marc format and add: U8kohauserP8kohapass at the start - connection expects those two fields then parses the rest as marc - you will need to create a user with those credentials and permissions to edit the catalogue - or substitute your own credentials > 4) For step 8 we get the following, is this to be expected? > > kohadev-koha@kohadevbox:koha((v24.05.06))$ cat bug_33418.test | nc -v > localhost 8888 > nc: connect to localhost (::1) port 8888 (tcp) failed: Connection refused > Connection to localhost (127.0.0.1) 8888 port [tcp/*] succeeded! > Unsuccessful requestkohadev-koha@kohadevbox:koha((v24.05.06))$ No, the request shoudl be successful, but perhaps the user wasn't created? I get: kohadev-koha@kohadevbox:koha(main)$ cat bug_33418.test | nc -v localhost 8888 nc: connect to localhost (::1) port 8888 (tcp) failed: Connection refused Connection to localhost (127.0.0.1) 8888 port [tcp/*] succeeded! Success. Batch number 2 - biblio record number 439 overlaid by import. http://kohadev-intra.myDNSname.org:8081/cgi-bin/koha/catalogue/detail.pl?biblionumber=439kohadev-koha@kohadevbox:koha(main)$
(In reply to Nick Clemens (kidclamp) from comment #12) > (In reply to Alex Buckley from comment #11) > > Hi, > > > > 1) Can you please give us a more detailed test plan? > Which parts are unclear? I should have mentioned creating a user with > kohauser/kohapass as the credentials, or use the credentials for an existing > account with the correct permissions > > > 2) Are we supposed to make the /etc/koha/sites/kohadev/connexion.cnf file? > Yes, create this file if it doesn't exist > > > 3) We noticed that there are some strange characters in the sample record > > for import file ( > > https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=174097 ), which > > is giving us the following output in step 6: > > > > kohadev-koha@kohadevbox:koha((v24.05.06))$ perl > > misc/bin/connexion_import_daemon.pl -c /etc/koha/sites/kohadev/connexion.cnf > > no mapping found for [0x90] at position 0 in ��96475 g0=ASCII_DEFAULT > > g1=EXTENDED_LATIN at /usr/share/perl5/MARC/Charset.pm line 308. > > no mapping found for [0x90] at position 2 in 8␞2024022215392 > > g0=ASCII_DEFAULT g1=EXTENDED_LATIN at /usr/share/perl5/MARC/Charset.pm line > > 308. > > no mapping found for [0x90] at position 4 in 4.0␞981015s1948 enk > > 000 g0=ASCII_DEFAULT g1=EXTENDED_LATIN at > > /usr/share/perl5/MARC/Charset.pm line 308. > > I think this may be from copying/pasting or the way it was downloaded - you > can simply export a single record from your koha in marc format and add: > U8kohauserP8kohapass > at the start - connection expects those two fields then parses the rest as > marc - you will need to create a user with those credentials and permissions > to edit the catalogue - or substitute your own credentials > > > > > 4) For step 8 we get the following, is this to be expected? > > > > kohadev-koha@kohadevbox:koha((v24.05.06))$ cat bug_33418.test | nc -v > > localhost 8888 > > nc: connect to localhost (::1) port 8888 (tcp) failed: Connection refused > > Connection to localhost (127.0.0.1) 8888 port [tcp/*] succeeded! > > Unsuccessful requestkohadev-koha@kohadevbox:koha((v24.05.06))$ > > No, the request shoudl be successful, but perhaps the user wasn't created? > I get: > kohadev-koha@kohadevbox:koha(main)$ cat bug_33418.test | nc -v localhost > 8888 > nc: connect to localhost (::1) port 8888 (tcp) failed: Connection refused > Connection to localhost (127.0.0.1) 8888 port [tcp/*] succeeded! > Success. Batch number 2 - biblio record number 439 overlaid by import. > > http://kohadev-intra.myDNSname.org:8081/cgi-bin/koha/catalogue/detail. > pl?biblionumber=439kohadev-koha@kohadevbox:koha(main)$ Sorry for the delay getting back to you Nick. Thank you for the following clarifications: - Creating a user with kohauser/kohapass as the credentials - Exporting a single record from Koha in marc format and adding 'U8kohauserP8kohapass' that did the trick, thank you! I have been able to successfully run through the test plan on 24.05.x and am backporting
Nice work everyone! Pushed to 24.05.x for 24.05.08
Not for 23.11.x
Bug fix, nothing to add/edit in the manual