Summary: | Add throttling / rate limiting to Koha | ||
---|---|---|---|
Product: | Koha | Reporter: | Kyle M Hall (khall) <kyle> |
Component: | Architecture, internals, and plumbing | Assignee: | David Cook <dcook> |
Status: | In Discussion --- | QA Contact: | Testopia <testopia> |
Severity: | enhancement | ||
Priority: | P5 - low | CC: | clemens.tubach, dcook, lukasz.koszyk, m.de.rooy, michaela.sieber, raphael.straub |
Version: | unspecified | ||
Hardware: | All | ||
OS: | All | ||
See Also: |
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31242 https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39435 |
||
GIT URL: | Change sponsored?: | --- | |
Patch complexity: | --- | Documentation contact: | |
Documentation submission: | Text to go in the release notes: | ||
Version(s) released in: | Circulation function: | ||
Attachments: | Bug 39109: Add Plack middleware to throttle requests |
Description
Kyle M Hall (khall)
2025-02-12 15:39:05 UTC
(In reply to Kyle M Hall (khall) from comment #0) > Botnets and web spiders can easily bring a Koha instance down. We should > implement optional rate limiting in Koha. It appears there are two plack > middlewares we could use: > https://metacpan.org/pod/Plack::Middleware::Throttle or > https://metacpan.org/pod/Plack::Middleware::Throttle::Lite > > Throttle supports memcached as a backend, but is more complex. Lite uses an > in-memory backend, but has fewer dependencies. I was inspired by Plack::Middleware::Throttle to write my own Koha::Middleware::Throttle. It uses memcached as the storage backend, and its expiring keys mechanism means it is self-managing, which is great. My version does use some local custom code for managing configuration in a more flexible way, so I'll need to strip that out, but it's still useful without it. Happy to share my work here. Created attachment 178130 [details] [review] Bug 39109: Add Plack middleware to throttle requests This patch includes a configurable Plack middleware which limits the number of requests for an IP address range for a moving time window. For example, 192.168.1.0/24 can only make 5 requests over the last 1 minute. Test plan: 1. vi /etc/koha/sites/kohadev/plack.psgi 2. Add the following to the /opac mount point (higher than Koha::Middlware::CSRF but lower than HTTPExceptions) : enable "+Koha::Middleware::Throttle", debug => 1, interface => 'opac', paths => [ '^/opac-search', ], allow_list => [ '192.168.1.0/24', ], request_threshold => 5, #Number of requests to trigger throttle last_n_minutes => 1, #Number of minutes to consider when aggregating request counts cidr_block => 24, #CIDR mask to use for determing range IP address belongs to (may only be 8, 16, 24, 32) ; 3. sudo koha-plack --restart kohadev 4. Do 6 requests to opac-search.pl within 1 minute and see the pretty "You've been throttled" page. I have other work to present an interactive challenge for users, but it's not currently compatible with this feature, nor ready to be shared just yet. I think a challenge page is better than a blocked page, as it gives legitimate human users an opportunity to get back to searching. But that's a whole other thing. I was inspired by Plack::Middleware::Throttle, but I found it to be too limited for our purposes. You'll likely notice limited use of Koha::Cache, and that's because we don't implement the "incr" method in Koha::Cache, which is vital for multi-process safety in this feature. -- I think there's lots of room for improvement with this feature, although it is limited in what it can do against distributed botnets and web spiders. That said, I just looked at some logs, and I saw it throttled a couple of spiders, so every little bit helps. Thanks for sharing this. We are currently under fire too :) I am still thinking about a challenge and some cookie to pass the challenge.. (In reply to David Cook from comment #4) > I have other work to present an interactive challenge for users, but it's > not currently compatible with this feature, nor ready to be shared just yet. > > I think a challenge page is better than a blocked page, as it gives > legitimate human users an opportunity to get back to searching. But that's a > whole other thing. If you submit, please keep me in cc. |