---- Reported by nicolas.morin@biblibre.com 2009-04-20 13:44:31 ---- if the opac-suggestions.pl?op=add page is accessible to anonymous users (AnonSuggestions syspref ON), we should provide a catpcha. Currently we don't, and spam coming through the form quickly forces the library to close down the option. I suggest http://search.cpan.org/dist/Captcha-reCAPTCHA/lib/Captcha/reCAPTCHA.pm It works well and has something to do with libraryland. ---- Additional Comments From mjr@ttllp.co.uk 2009-04-20 13:57:14 ---- Please add anti-spam tests like blogspam.net or even simple heuristics to defeat spam. A sample of the spam you're getting would be useful. Please do not add reCAPTCHA. It discriminates against humans with visual or hearing impairments. Retitling bug. ---- Additional Comments From jeanandre.santoni@biblibre.com 2009-06-16 14:12:15 ---- reCAPTCHA only discriminates against humans with both visual and hearing impairments. ---- Additional Comments From mjr@ttllp.co.uk 2009-06-16 14:20:10 ---- Hearing impairments sufficient to prevent reCAPTCHA use are pretty common (estimates over 40% are common). Please implement anti-spam in preference to reCAPTCHA. CAPTCHAs are not anti-spam and I suspect most implementations violate disability discrimination laws. ---- Additional Comments From jeanandre.santoni@biblibre.com 2009-06-17 15:11:37 ---- Blogspam.net doesn't seem to work so well. Take a look at my log (I Data::Dumped my request and their result): $VAR1 = 'comment'; $VAR2 = 'La soupe à la fourchette Jean-Angleade 1987 Poche '; $VAR3 = 'ip'; $VAR4 = '88.191.19.91'; SPAM:SpamBayes at /home/kivutar/Workspace/koha30x/C4/Suggestions.pm line 349. According to blogspam.net, "La soupe à la fourchette Jean-Angleade 1987 Poche" is a spam... Do you know another service? Or maybe I just used it wrong. ---- Additional Comments From mjr@ttllp.co.uk 2009-06-18 14:31:58 ---- I asked about this and was told it's essentially a result of using French-language data with one of the plugins that's trained with English-language ham data: "I train the bayasian filter using results that are known-good or known-spam, but in both cases the training is with English text. I'd never considered the use of foreign language to be an issue, but I can easily imagine if the corpus is always trained with English that French wouldn't be seen, and is more likely to weigh in as spam. If this is a common occurrence then the caller should disable the bayasian plugin when they submit their comments for testing, see here for details of how to do that: http://blogspam.net/api/testComment.html (You'll want to disable the plugin called "bayasian".)" Alternatively, Francophones may like to run their own spam filter, trained with French-language ham. (By the way, did you know that the documents used for reCAPTCHA are all taken from US sources at the moment? I hear it's great fun for UKUS and NZUS spelling differences.) ---- Additional Comments From jeanandre.santoni@biblibre.com 2009-06-22 11:36:32 ---- Created an attachment Bugfix Bugfix ---- Additional Comments From mjr@ttllp.co.uk 2009-06-22 11:43:35 ---- Mollom seems to discriminate against visually-impaired users with non-CAPTCHA eyetests if not thought to be ham and the free service only allows 100 comments per day - see http://mollom.com/pricing Therefore, integrating this Mollom patch would cause both a new accessibility error and open an obvious denial of service attack opportunity. Please make this a contrib only. Also, it's spelt "anonymous" not "anonimous". ---- Additional Comments From jeanandre.santoni@biblibre.com 2009-06-22 12:17:43 ---- The use I made of Mollom does not discriminate against visually-impaired users. It is just a filter like blogspam.net. Except that it works better. Can you tell me more about the denial of service attack opportunity please? ---- Additional Comments From gmcharlt@gmail.com 2009-06-29 15:12:46 ---- Comment on proposed patch: My main concern with this patch is that there needs to be an abstraction layer, no matter how thin, for spam filtering before we start adding in support for specific services like Mollom. A simple class in C4 that can delegate a spam/ham decision should be sufficient. We have a big enough mess with the current state of support for enhanced content providers without any kind of abstraction layer, so I'd like to take this opportunity for us to not fall into a similar trap. --- Bug imported by chris@bigballofwax.co.nz 2010-05-21 01:06 UTC --- This bug was previously known as _bug_ 3144 at http://bugs.koha.org/cgi-bin/bugzilla3/show_bug.cgi?id=3144 Imported an attachment (id=1127) Actual time not defined. Setting to 0.0 The original reporter of this bug does not have an account here. Reassigning to the person who moved it here: chris@bigballofwax.co.nz. Previous reporter was nicolas.morin@biblibre.com. The original assignee of this bug does not have an account here. Reassigning to the default assignee for the component, chris@bigballofwax.co.nz. Previous assignee was jeanandre.santoni@biblibre.com. Bug reassigned, setting status to "NEW". Previous status was "ASSIGNED". CC member mjr@ttllp.co.uk does not have an account here The original submitter of attachment 1127 [details] [review] is unknown. Reassigning to the person who moved it here: chris@bigballofwax.co.nz.
I'm removing "PATCH-sent" status from this bug since the correct resolution is still under discussion.
Restoring where "Previous assignee was mjr@ttllp.co.uk"
Noting that a Koha user recently reported an instance of purchase suggestion spam: http://lists.katipo.co.nz/pipermail/koha/2013-July/036898.html
Hi everyone I have been investigating this issue today for one of Calyx's clients. I decided to do a bit of reading up on the issue of spam protection. reCAPTCHA was the first solution that came to mind, however valid points about accessibility issues have already been made. What about the possibility of implementing a negative captcha? That way OPACs remain accessible. It would also mean we don't have to deal with the false positive detection of suggestions (as with analytical spam detection). The only downside I can see with this would be a bruteforce tool that tries ignoring particular fields or a custom coded spam bot (which seems very unlikely considering the value of the target). I also like the http://visualcaptcha.net/ project. Chris
A negative captcha seems to be the current best practice, I vote we go with that.
(In reply to Chris Cormack from comment #5) > A negative captcha seems to be the current best practice, I vote we go with > that. Agreed.
(In reply to Galen Charlton from comment #6) > (In reply to Chris Cormack from comment #5) > > A negative captcha seems to be the current best practice, I vote we go with > > that. > > Agreed. A client has agreed to fund this. We'll proceed with it.
Created attachment 28504 [details] [review] Bug 3144 - anti-spam for opac-suggestions 1/ 1st, enable 'suggestion' and 'AnonSuggestions' sysprefs 2/ disable javascript in your web-browser 3/ attempt to add a suggestion in OPAC, but leave the 'secret' field empty. - adding the suggestion should succeed http://yourkoha.org/cgi-bin/koha/opac-suggestions.pl 4/ attempt to add another suggestion, and populate 'secret' field - adding the suggestion should fail 5/ enable javascript in your web-browser 6/ attempt to add a suggestion, NOTE: the 'secret' field should now be hidden via javascript - adding the suggestion should succeed
Created attachment 28505 [details] [review] Bug 3144 - anti-spam for opac-suggestions this patch adds a negative-captcha feature to the purchase suggestions form some info for the curious... http://nedbatchelder.com/text/stopbots.html https://github.com/subwindow/negative-captcha http://www.rubydoc.info/github/subwindow/negative-captcha/frames to test this patch... 1/ apply patch 2/ enable 'suggestion' and 'AnonSuggestions' sysprefs 3/ attempt to add a 'purchase suggestion' in OPAC adding the suggestion should succeed 4/ edit the opac-suggestions.tt file, to reveal the secret field replace line 130, from... <li id="negcap" style="position: absolute; left: -2000px;"> to ... <li id="negcap"> 5/ attempt to add another suggestion, and populate the 'negcap' field adding the suggestion should fail
Created attachment 28506 [details] suggestions form, with hidden 'negcap' field showing
Created attachment 28507 [details] [review] Bug 3144 - anti-spam for opac-suggestions this patch adds a negative-captcha feature to the purchase suggestions form some info for the curious... http://nedbatchelder.com/text/stopbots.html https://github.com/subwindow/negative-captcha http://www.rubydoc.info/github/subwindow/negative-captcha/frames to test this patch... 1/ apply patch 2/ enable 'suggestion' and 'AnonSuggestions' sysprefs 3/ attempt to add a 'purchase suggestion' in OPAC adding the suggestion should succeed 4/ edit the koha-tmpl/opac-tmpl/bootstrap/en/modules/opac-suggestions.tt file, to reveal the hidden 'negcap' field replace line 87 opac-suggestions.tt, from... <li id="negcap" style="position: absolute; left: -2000px;"> to ... <li id="negcap"> 5/ attempt to add another suggestion, and populate the 'negcap' field adding the suggestion should fail
Created attachment 28508 [details] suggestions form, with hidden 'negcap' field showing
patch updated for bootstrap, not prog theme :)
Works great. I recommend updating the other themes with this function as well. Christopher
Patch tested with a sandbox, by Christopher Brannon <cbrannon@cdalibrary.org>
Created attachment 28667 [details] [review] Bug 3144 - anti-spam for opac-suggestions this patch adds a negative-captcha feature to the purchase suggestions form some info for the curious... http://nedbatchelder.com/text/stopbots.html https://github.com/subwindow/negative-captcha http://www.rubydoc.info/github/subwindow/negative-captcha/frames to test this patch... 1/ apply patch 2/ enable 'suggestion' and 'AnonSuggestions' sysprefs 3/ attempt to add a 'purchase suggestion' in OPAC adding the suggestion should succeed 4/ edit the koha-tmpl/opac-tmpl/bootstrap/en/modules/opac-suggestions.tt file, to reveal the hidden 'negcap' field replace line 87 opac-suggestions.tt, from... <li id="negcap" style="position: absolute; left: -2000px;"> to ... <li id="negcap"> 5/ attempt to add another suggestion, and populate the 'negcap' field adding the suggestion should fail Signed-off-by: Christopher Brannon <cbrannon@cdalibrary.org>
(In reply to Christopher Brannon from comment #14) > Works great. I recommend updating the other themes with this function as > well. > > Christopher The other themes were deprecated in 3.14.0 and will be removed in 3.18.0 So you would probably have to fund someone to backport this to them if you wanted.
Do you think it should be communicated to the user that the submission failed? I know it is highly unlikely, but in the off chance that it is a human, and something goes wrong, should something be in place? Christopher
(In reply to Christopher Brannon from comment #18) > Do you think it should be communicated to the user that the submission > failed? I know it is highly unlikely, but in the off chance that it is a > human, and something goes wrong, should something be in place? > Perhaps, however id be worried that telling them it failed might make the bot try again and again.
Created attachment 28701 [details] [review] Bug 3144 - anti-spam for opac-suggestions this patch adds a negative-captcha feature to the purchase suggestions form some info for the curious... http://nedbatchelder.com/text/stopbots.html https://github.com/subwindow/negative-captcha http://www.rubydoc.info/github/subwindow/negative-captcha/frames to test this patch... 1/ apply patch 2/ enable 'suggestion' and 'AnonSuggestions' sysprefs 3/ attempt to add a 'purchase suggestion' in OPAC adding the suggestion should succeed 4/ edit the koha-tmpl/opac-tmpl/bootstrap/en/modules/opac-suggestions.tt file, to reveal the hidden 'negcap' field replace line 87 opac-suggestions.tt, from... <li id="negcap" style="position: absolute; left: -2000px;"> to ... <li id="negcap"> 5/ attempt to add another suggestion, and populate the 'negcap' field adding the suggestion should fail Signed-off-by: Christopher Brannon <cbrannon@cdalibrary.org> Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com> Amended patch: Remove the bug report number in the comment since git log cans provide the same result.
Simple patch, easy to test, pushed to Master
(In reply to Chris Cormack from comment #21) > Simple patch, easy to test, pushed to Master This enhancement has been sponsored by Professional Support Coordinators' Alliance, which provides professional support and training to the child care and early learning sector in Australia.
Given that some libraries have been significantly affected by purchase suggestion spam and consequently have not been able to use AnonSuggestions, I'm considering this a bugfix. Therefore, pushed to 3.16.x for inclusion in 3.16.1.
Chatted with Chris Cormack, Chris Vella, and Mason about this a bit on IRC. Screen readers will still read the 'negcap' field, which is suboptimal. However, even if a screenreader does read the field, hopefully people will still leave this blank. Perhaps it might be an idea to include a more obvious label name like "do not fill me in". Anyway, just leaving some links here. Chris Vella found the one about "aria-hidden". That could be really good to add. Relevant links: http://www.w3.org/TR/wai-aria/states_and_properties#aria-hidden http://www.visionaustralia.org/living-with-low-vision/learning-to-live-independently/using-technology-and-computers/blog---accessibility-and-assistive-technology-blog/blog/accessibility-blog/2014/12/09/effective-alternatives-to-inaccessible-captchas http://webaim.org/techniques/css/invisiblecontent/#absolutepositioning
A human being might still misuse the feature - I think it should be a choice for the library, so they can turn off 'non logged in cart sending'.
Last comment was meant for another bug... bug 4274
Hmm, I think I've found a small bug here, patch coming...
I think that this was working for a time, but it appears that some bots may have wised up... I'm noticing spam coming through once again, which I assume means that someone is having their bot ignore the "negcap" field or fields that are located off screen. Both would be easy to do. We're going to look at blocking some IP addresses to start, but this one is a distributed spamming of the system so that will likely only be partially successful. I have a couple curl commands that I could use to easily spam opac-suggestions.pl...
You know... we could do a text-based positive captcha. That should be accessible. Something like... "write the third word from the previous paragraph in the following box". A human should be able to understand that instruction, and it should require fairly sophisticated natural language processing for a bot to figure out.
Ideally, it wouldn't always be the third word. It would sometimes be the first word or the last word of the paragraph. Something like that...
Potentially the bot could be trying input forms at random, or more advanced and only reading through the visible elements of the page. Rather than go down the positive captcha route yet. We could attempt to make this negcaptcha smarter. One idea could be to IP ban any user who submits content to that form (I don't see any reason why a normal user would). We could also have the field name randomise per instance.
The self registration ( cgi-bin/koha/opac-memberentry.pl ) has already a text based captcha with a random string: "Please type the following characters into the preceding box: ODXZX" I think it would be good to re-use technique/code at all places where we have 'human user input'.
(In reply to Mason James from comment #27) > Hmm, I think I've found a small bug here, patch coming... update: i added the patch to a new SEC bug https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=15035
(In reply to David Cook from comment #28) > I think that this was working for a time, but it appears that some bots may > have wised up... > hiya, are you running this follow-up patch? https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=15035
(In reply to Mason James from comment #34) > hiya, are you running this follow-up patch? > > https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=15035 Alas, I don't have access to see that security bug :/.
(In reply to Christopher Vella from comment #31) > Potentially the bot could be trying input forms at random, or more advanced > and only reading through the visible elements of the page. > Yeah, I think there are a few different things it could be. I'd love to know what data is being sent exactly. > Rather than go down the positive captcha route yet. We could attempt to make > this negcaptcha smarter. Totally agree. I'm absolutely on board for that. > One idea could be to IP ban any user who submits > content to that form (I don't see any reason why a normal user would). I'm not sure I understand what you mean here. When the user fills out the form, their browser would be POSTing the data to the Koha server, so it would be something a normal user would do. > We could also have the field name randomise per instance. Maybe. I'd be interested to see how other people are solving this problem out in the world.
(In reply to David Cook from comment #35) > (In reply to Mason James from comment #34) > > hiya, are you running this follow-up patch? > > > > https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=15035 > > Alas, I don't have access to see that security bug :/. Fixed :)
> > One idea could be to IP ban any user who submits > > content to that form (I don't see any reason why a normal user would). > > I'm not sure I understand what you mean here. When the user fills out the > form, their browser would be POSTing the data to the Koha server, so it > would be something a normal user would do. Normal users shouldn't be submitting any data to the negcap field (because its not visible). Therefore only spammers should be submitting data too it. If we introduced a banning system, their subsequent spamming attempts would be stopped.
(In reply to Christopher Vella from comment #38) > Normal users shouldn't be submitting any data to the negcap field (because > its not visible). Therefore only spammers should be submitting data too it. > If we introduced a banning system, their subsequent spamming attempts would > be stopped. Ah yes. I understand you now. Yeah, that would lower the amount of spam requests you're getting. It seems that we're getting POSTs without the negcap field filled in, which is how we're still getting spam suggestions, but with your idea they may have been banned before they figured that out...
(In reply to Mason James from comment #34) > hiya, are you running this follow-up patch? > > https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=15035 Not running the follow-up patch, but that doesn't appear to be an issue for 3.14 :/