Bug 37060 - KOCT cannot send circulation data due to missing CSRF token
Summary: KOCT cannot send circulation data due to missing CSRF token
Status: NEW
Alias: None
Product: Koha
Classification: Unclassified
Component: Circulation (show other bugs)
Version: Main
Hardware: All All
: P5 - low normal
Assignee: Matthias Meusburger
QA Contact: Testopia
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2024-06-10 12:07 UTC by Jan Kissig
Modified: 2025-04-02 14:57 UTC (History)
14 users (show)

See Also:
Change sponsored?: ---
Patch complexity: ---
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:
Circulation function:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Jan Kissig 2024-06-10 12:07:47 UTC
Just tried using the firefox offline circulation tool (koct) and ran into these console errors:

 ==> /var/log/koha/kohadev/plack-intranet-error.log <==
[2024/06/10 11:55:12] [WARN] Programming error - No CSRF token passed for POST http://localhost:8081/intranet/offline_circ/service.pl (referer: No referer) at /kohadevbox/koha/Koha/Middleware/CSRF.pm line 82.


To reproduce:
a) install firefox addon linked in http://localhost:8081/cgi-bin/koha/circ/circulation-home.pl 
b) in the plugin settings:
  server: http://localhost:8081
  Login: koha
  Password: koha
c) test login
d) check logs for 

[WARN] Programming error - No CSRF token passed for POST http://localhost:8081/intranet/offline_circ/service.pl (referer: No referer) at /kohadevbox/koha/Koha/Middleware/CSRF.pm line 82.
Comment 1 David Cook 2024-06-11 05:01:01 UTC
Looks like BibLibre might maintain that tool? 

Looks like Bywater will need to update their desktop app too.
Comment 2 David Cook 2024-06-11 05:02:10 UTC
Note that a short-term solution might be for them to use the SVC API.

(I don't include links anymore as Bugzilla always autobans me when I do, but you can find out more on the wiki for page "Koha /svc/ HTTP API")
Comment 3 Katrin Fischer 2024-06-25 09:31:22 UTC
Adding Claire, Matts and Frido to the bug, as they are listed here:
https://addons.mozilla.org/de/firefox/addon/koct/

And Kyle for the desktop application.

I believe the KOCT plugin is still used in a lot of libraries, I know ours do. 

I am not sure the svc is a great option, do we have others?
Comment 4 Jan Kissig 2024-07-15 11:49:19 UTC
With Bug 24401 (checkin endpoint) and Bug 37253 (checkout via barcode) it would be possible to use the API for KOCT.
Comment 5 David Cook 2024-10-14 22:44:04 UTC
So I'm not 100% sure what I meant by my earlier comments.

But what I should've meant was that the author of the Firefox add-on could use the SVC API to obtain a CSRF token and then do its existing POST.

Happy to collaborate with BibLibre on this one, if they want any clarification on what I mean here.
Comment 6 Matthias Meusburger 2024-10-16 08:31:05 UTC
Thanks David for the feedback!

You're thinking about https://wiki.koha-community.org/wiki/Koha_/svc/_HTTP_API#Changes_coming_in_Koha_24.05 ?

"This initial request generates a CGISESSID cookie linked to an anonymous session and provides a Csrf-Token in a response header."

Correct?
Comment 7 David Cook 2024-10-17 00:30:52 UTC
(In reply to Matthias Meusburger from comment #6)
> Thanks David for the feedback!
> 
> You're thinking about
> https://wiki.koha-community.org/wiki/Koha_/svc/
> _HTTP_API#Changes_coming_in_Koha_24.05 ?
> 
> "This initial request generates a CGISESSID cookie linked to an anonymous
> session and provides a Csrf-Token in a response header."
> 
> Correct?

More or less!

Basically, you do that first anonymous GET request to get the anonymous CGISESSID cookie and a Csrf-Token (via the HTTP response headers):

e.g. curl -v 'http://localhost:8081/cgi-bin/koha/svc/authentication' --cookie-jar /tmp/test.cookies

Then, using that same cookie and that Csrf-Token, you'll probably want to login using that endpoint as well (I'll explain more in a moment):

e.g. curl -v -H "Content-Type: application/x-www-form-urlencoded" -H "Csrf-Token: 4dda36756ec4f7ac178fb5500e1873b6f50cf9bc,699bc3e9fbb01ca7282bc8d847d8a018e696d327,1716342462" -XPOST -v 'http://localhost:8081/cgi-bin/koha/svc/authentication' -d "login_userid=koha&login_password=koha" --cookie /tmp/test.cookies --cookie-jar /tmp/test.cookies

Now the reason you'll probably want to do this is because you'll get a *NEW* authenticated CGISESSID cookie, and a *NEW* CSRF token.

You can use these *NEW* credentials for as long as the cookie and token are valid (the token is valid for up to 8 hours if I recall correctly). 

If you skip this step, you'll probably get stuck, since "offline/service.pl" doesn't appear to return a cookie or Csrf-Token. In other words, if you skip this login step, you'd have to repeat the 1st step for every "offline_circ/service.pl" request, and you don't want to do that.

Anyway, then, you should be able to do as many API requests as you want to that "offline_circ/service.pl" script. 

--

Let me know if that doesn't make sense. 

You can look at "./misc/migration_tools/koha-svc.pl" for some examples of what this looks like in practice. 

If you have a public git for it, I could also take a little look to make more in-context comments.
Comment 8 David Cook 2024-10-21 23:52:20 UTC
Matts, can we marked this one as assigned to you?
Comment 9 Matthias Meusburger 2024-10-23 06:26:07 UTC
I've assigned this ticket to myself, yes.
Comment 10 Todd Goatley 2024-12-27 23:48:00 UTC
HI Matthias!
Apologies, I'm sure you're busy and we're in the holiday season but there have been libraries requesting information on the status of this bug. Any update/timeline would be greatly appreciated. 
Thank you, sir!
Comment 11 Matthias Meusburger 2024-12-30 08:26:28 UTC
Hi!

So, the current status is: I already gave it a shot a few weeks back, but didn't succeed at implement David's solution (which works perfectly well using command line) in KOCT. 

So I definitely have to have another look at it.

If anyone has a working fix, we accept pull requests here: https://git.biblibre.com/biblibre/koct-webext

And, I know it is not ideal, but in the meantime, users can still export data (.koc file) from KOCT and upload it in the offline circulation tool in Koha.
Comment 12 David Cook 2024-12-30 22:32:15 UTC
(In reply to Matthias Meusburger from comment #11)
> So, the current status is: I already gave it a shot a few weeks back, but
> didn't succeed at implement David's solution (which works perfectly well
> using command line) in KOCT. 

Are you using LWP::UserAgent? If so, double-check if you're setting a cookie_jar. From memory, LWP::UserAgent doesn't retain cookies in between requests, so you'll need to remember to set "cookie_jar" to {} either in the constructor or using a method after object creation.

Just guessing that might be the gotcha.
Comment 13 David Cook 2024-12-30 22:50:29 UTC
(In reply to Matthias Meusburger from comment #11)
> So, the current status is: I already gave it a shot a few weeks back, but
> didn't succeed at implement David's solution (which works perfectly well
> using command line) in KOCT. 
>
> If anyone has a working fix, we accept pull requests here:
> https://git.biblibre.com/biblibre/koct-webext

Ah, I forgot it was a browser extension and not a Perl script. That's extra interesting...

I'd have to see your patches, but 1 problem would be that "userid" and "password" have become "login_userid" and "login_password" in Koha, so the current koct.js won't authenticate correctly anyway.

--

But yeah looking through the koct.js code and offline_circ/service.pl... the Koha session cookie is probably where the problem is. 

It's been a long time since I wrote a browser extension, so I don't recall the intricacies of CORS requests in them. Might need to set 'xhr.withCredentials = true' to get the cookie response for the SVC API stuff. 

--

Alas, I won't have time to look at this for a month at least :/.
Comment 14 Jan Kissig 2025-01-03 19:23:58 UTC
Hi there, I tried to implement what David said but somehow the authenticated cookie and the CRSF-token are bound together, and when I loose that token (but keep the session), there seems no chance of getting a valid token again. 

Wiki says:
If you lose it for whatever reason, you can get a new Csrf-Token by using your authenticated cookie and sending a GET to /cgi-bin/koha/svc/authentication like you did in the first step. 
---

The token I received by GET /cgi-bin/koha/svc/authentication will always throw "wrong_csrf_token" so I build a workaround by logging out if GET /cgi-bin/koha/svc/authentication returns a valid session (<status>ok</status>)

I uploaded a fixed version to https://gitlab.com/bibliothekTHWildau/koct-webext as I don't know if a pull request would work as I have no account on your git server.

If someone wants to try with firefox (I used a german firefox , so maybe the naming is a bit different): 

- go to addons and themes
- click the cog wheel and debug addons
- load temporary add on
- navigate to the downloaded extension and double click manifest.json
Comment 15 David Cook 2025-01-05 23:26:07 UTC
(In reply to Jan Kissig from comment #14)
> Hi there, I tried to implement what David said but somehow the authenticated
> cookie and the CRSF-token are bound together, and when I loose that token
> (but keep the session), there seems no chance of getting a valid token
> again. 

So the cookie contains the session ID, and the CSRF token is bound to that session ID. 

> Wiki says:
> If you lose it for whatever reason, you can get a new Csrf-Token by using
> your authenticated cookie and sending a GET to
> /cgi-bin/koha/svc/authentication like you did in the first step. 
> ---
> 
> The token I received by GET /cgi-bin/koha/svc/authentication will always
> throw "wrong_csrf_token" so I build a workaround by logging out if GET
> /cgi-bin/koha/svc/authentication returns a valid session
> (<status>ok</status>)

I've just confirmed your problem using curl, so I'll look into that. Something very odd going on here, especially since I'm sure this used to work...
Comment 16 David Cook 2025-01-05 23:37:32 UTC
(In reply to David Cook from comment #15)
> I've just confirmed your problem using curl, so I'll look into that.
> Something very odd going on here, especially since I'm sure this used to
> work...

So it looks like a bug in C4::Auth::check_api_auth, but it looks like a bug that has existed for years, so maybe this only worked theoretically in the past...

Basically, C4::Auth::check_api_auth() sometimes returns $sessionID and sometimes it returns $session (like in the case where you already have an authenticated cookie), even though it's supposed to always return $sessionID.
Comment 17 David Cook 2025-01-06 00:00:47 UTC
I've posted a patch to bug 38826, but yeah... I'd say have to use a workaround in the meantime. 

Btw, I think instead of hitting '/cgi-bin/koha/mainpage.pl?logout.x=1' you could use 
"credentials: 'omit'" to force a new login, but I suppose it's all the same in the end.
Comment 18 Todd Goatley 2025-01-06 17:49:57 UTC
(In reply to David Cook from comment #17)
> I've posted a patch to bug 38826, but yeah... I'd say have to use a
> workaround in the meantime. 
> 
> Btw, I think instead of hitting '/cgi-bin/koha/mainpage.pl?logout.x=1' you
> could use 
> "credentials: 'omit'" to force a new login, but I suppose it's all the same
> in the end.

Thank you David and Matthias!
Comment 19 Jan Kissig 2025-01-06 19:14:01 UTC
(In reply to David Cook from comment #17)
> I've posted a patch to bug 38826, but yeah... I'd say have to use a
> workaround in the meantime. 
> 
> Btw, I think instead of hitting '/cgi-bin/koha/mainpage.pl?logout.x=1' you
> could use 
> "credentials: 'omit'" to force a new login, but I suppose it's all the same
> in the end.

Hi David, 
thx for filing 38826. This changed the token handling a lot, as no logout is necessary anymore. I fixed the web extension based on bug_38826 and uploaded a new version (see #14)
Comment 20 David Cook 2025-01-06 22:09:49 UTC
(In reply to Todd Goatley from comment #18)
> Thank you David and Matthias!

Thank you Jan too :D

(In reply to Jan Kissig from comment #19)
> thx for filing 38826. This changed the token handling a lot, as no logout is
> necessary anymore. I fixed the web extension based on bug_38826 and uploaded
> a new version (see #14)

Thanks for testing 38826. Hopefully we can get the patch into Koha soon.
Comment 21 David Cook 2025-01-13 01:16:06 UTC
Bug 38826 has been pushed for 25.05.00. I'm asking for it to be backported to 24.11 and 24.05 as well, since those are the other versions that use anti-CSRF tokens.
Comment 22 Matthias Meusburger 2025-01-16 09:41:49 UTC
Thanks Jan for your patches.

I have tested them successfully and will update KOCT on addons.mozilla.org as soon as possible.
Comment 23 Jan Kissig 2025-01-16 10:03:14 UTC
(In reply to Matthias Meusburger from comment #22)
> Thanks Jan for your patches.
> 
> I have tested them successfully and will update KOCT on addons.mozilla.org
> as soon as possible.

Hey Matthias,

I did test it for 24.05 and main but I think it will not work with older, but still used, koha versions. As add-ons can automatically be updated this would break things for older kohas.
Comment 24 Matthias Meusburger 2025-01-16 10:17:15 UTC
You're right. We need a patch that detects the Koha version and adapt KOCT behavior accordingly, or a configuration option to set Koha version if we cannot detect it automatically.
Comment 25 Matthias Meusburger 2025-02-03 16:15:01 UTC
Hi,

I've added a github repository for KOCT to make future contributions easier: https://github.com/biblibre/firefox-addon-koct

I've also added a patch that allows the user to choose Koha's version in the options in order to adapt koct's behavior (ie: with or without csrf).

I've tested it successfully on pre-24.05 and main.

It should work with all versions once Bug 38826 is backported to 24.11 and 24.05.

Could someone test this before I make a new release of the plugin? Thanks.

(The plugin can be tested by following the steps described on comment 14: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=37060#c14 )
Comment 26 Eric Bégin 2025-03-31 08:47:11 UTC
No more Missing CSRF token error !

However, the "Send data to Koha" commit option seems broken.
Comment 27 Eric Bégin 2025-03-31 09:06:06 UTC
It works!  My problem was that I was not connected on the right branch.
Comment 28 Matthias Meusburger 2025-03-31 09:31:21 UTC
Thanks Éric for testing, thanks everyone involved in this BZ.

The new 4.0.21 version of the KOCT firefox add-on has been submitted to addons.mozilla.org and is waiting for approval.
Comment 29 Matthias Meusburger 2025-03-31 09:36:09 UTC
Version 4.0.21 has been approved and is now available at https://addons.mozilla.org/fr/firefox/addon/koct/
Comment 30 Kyle M Hall (khall) 2025-04-02 13:12:12 UTC
(In reply to Matthias Meusburger from comment #29)
> Version 4.0.21 has been approved and is now available at
> https://addons.mozilla.org/fr/firefox/addon/koct/

We have a partner running Koha 24.05. When we try to configure the plugin we get the "Forbidden" message.

I've selected Koha version >= 24.05 in the settings.

I've verified this myself with the latest plugin version ( 0.4.21 ).

Any idea what's going on?

Here are the logs:

==> /var/log/koha/library/plack.log <==
24.165.212.106 - - [02/Apr/2025:09:05:18 -0400] "GET /intranet_svc/authentication HTTP/1.1" 200 84 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:136.0) Gecko/20100101 Firefox/136.0"

==> /var/log/koha/library/intranet-access.log <==
staff.library.bywatersolutions.com:80 24.165.212.106 - - [02/Apr/2025:09:05:18 -0400] "GET /cgi-bin/koha/svc/authentication HTTP/1.1" 200 504 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:136.0) Gecko/20100101 Firefox/136.0"

==> /var/log/koha/library/plack-intranet-error.log <==
[2025/04/02 09:05:18] [WARN] wrong_csrf_token at /usr/share/koha/lib/Koha/Middleware/CSRF.pm line 97.

==> /var/log/koha/library/plack.log <==
24.165.212.106 - - [02/Apr/2025:09:05:18 -0400] "POST /intranet_svc/authentication HTTP/1.1" 403 16 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:136.0) Gecko/20100101 Firefox/136.0"

==> /var/log/koha/library/intranet-access.log <==
staff.library.bywatersolutions.com:80 24.165.212.106 - - [02/Apr/2025:09:05:18 -0400] "POST /cgi-bin/koha/svc/authentication HTTP/1.1" 403 197 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:136.0) Gecko/20100101 Firefox/136.0"
Comment 31 Matthias Meusburger 2025-04-02 14:36:19 UTC
Hi Kyle!

I'm not sure why you have this error, but I just tested on a fresh 24.05.08 koha, with koct 0.4.21 and it worked.
Comment 32 Kyle M Hall (khall) 2025-04-02 14:57:43 UTC
(In reply to Matthias Meusburger from comment #31)
> Hi Kyle!
> 
> I'm not sure why you have this error, but I just tested on a fresh 24.05.08
> koha, with koct 0.4.21 and it worked.

Thanks for checking! I'll have to dig deeper then!