Forums

Unfortunately no one can be told what FluxBB is - you have to see it for yourself.

You are not logged in.

#26 2010-05-06 16:07:03

quy
Administrator
From: California
Registered: 2008-05-09
Posts: 926

Re: Anti Spam in core

Bots signup for linkage in signature and website fields. Lets remove this incentive. Impose a minimum of 10 undeleted postings and only after 30 days before the signature and website URL will appear in profile page, and in postings. Would this work?

Offline

#27 2010-05-06 16:19:15

Paul
Developer
From: Wales, UK
Registered: 2008-04-27
Posts: 1,653

Re: Anti Spam in core

Isn't there something that can be done as regards speed of form completion/submit on the basis that a bot can complete the process faster than is humanly possible.


The only thing worse than finding a bug is knowing I created it in the first place.

Offline

#28 2010-05-06 16:24:59

damaxxed
Member
From: Germany
Registered: 2008-05-16
Posts: 353

Re: Anti Spam in core

Paul wrote:

Isn't there something that can be done as regards speed of form completion/submit on the basis that a bot can complete the process faster than is humanly possible.

Sounds good. What about the following limits?

Registration only possible, 30 seconds after viewing registration page. Only 2 registrations per IP per day (?)

Offline

#29 2010-05-06 16:33:00

Smartys
Former Developer
Registered: 2008-04-27
Posts: 3,139
Website

Re: Anti Spam in core

damaxxed wrote:
Paul wrote:

Isn't there something that can be done as regards speed of form completion/submit on the basis that a bot can complete the process faster than is humanly possible.

Sounds good. What about the following limits?

Registration only possible, 30 seconds after viewing registration page. Only 2 registrations per IP per day (?)

1. It doesn't take me 30 seconds to fill in a form.
2. Adding something like that would require keeping track of "state" in some form (ie: sessions) which FluxBB currently does not do.

Offline

#30 2010-05-06 16:38:40

damaxxed
Member
From: Germany
Registered: 2008-05-16
Posts: 353

Re: Anti Spam in core

Smartys wrote:

1. It doesn't take me 30 seconds to fill in a form.

Well, then we have a bot + Smartys protection wink

Smartys wrote:

2. Adding something like that would require keeping track of "state" in some form (ie: sessions) which FluxBB currently does not do.

AFAIK FluxBB stores the registration time and ip in the database, so a session is unneeded?

Offline

#31 2010-05-06 16:39:58

Reines
Administrator
From: Scotland
Registered: 2008-05-11
Posts: 3,197
Website

Re: Anti Spam in core

Doesn't take me 30 seconds to fill it out either tongue I assume Smartys was refering to telling how long between the register form being shown and submit, now your second idea.

Offline

#32 2010-05-06 16:40:41

Smartys
Former Developer
Registered: 2008-04-27
Posts: 3,139
Website

Re: Anti Spam in core

damaxxed wrote:
Smartys wrote:

1. It doesn't take me 30 seconds to fill in a form.

Well, then we have a bot + Smartys protection wink

Smartys wrote:

2. Adding something like that would require keeping track of "state" in some form (ie: sessions) which FluxBB currently does not do.

AFAIK FluxBB stores the registration time and ip in the database, so a session is unneeded?

Sorry, I was talking about protecting a form from being submitted too quickly.
Changing the registration limit from once/hour to twice/day may make sense for some forums, but I don't think it should be the default. Not enough people use static IPs: AOL users in particular would be hamstrung.

Offline

#33 2010-05-06 17:04:24

ridgerunner
Member
Registered: 2008-06-24
Posts: 183
Website

Re: Anti Spam in core

Why can't you just supply a "real" question on the registration form that only a human can answer?

The question would be a unique, administer created question, that no two forums would have in common. There could actually be a set of questions & answers that would be set up and maintained by the administrator and one out of the set would be chosen randomly for each registration attempt.

Also a time limit of 60 seconds or so between failed registration attempts would seem appropriate.

Offline

#34 2010-05-06 17:37:37

MattF
Member
From: South Yorkshire, England
Registered: 2008-05-06
Posts: 1,233
Website

Re: Anti Spam in core

Not all spam registrations are from bots.


Screw the chavs and God save the Queen!

Offline

#35 2010-05-06 17:58:30

Reines
Administrator
From: Scotland
Registered: 2008-05-11
Posts: 3,197
Website

Re: Anti Spam in core

ridgerunner wrote:

Why can't you just supply a "real" question on the registration form that only a human can answer?

Well it does work quite well, but a totally transparent solution would be far far nicer IMO. It may be though that these transparent solutions will only work short term, until someone writes a bot to cope with them.

We seem to have quite a few different types of bots hitting us from what I can tell from logs:

  1. Some appear to simply be posting the expected data to register.php without any regard to what form input's are actually there. These are caught by the honeypot, but also the fact the real username field has been renamed means they are submitting a blank username, and hence not getting through.

  2. Some are simply filling out all input fields and submitting. These are submitting a valid username in the new username field and oddly even seem to cope with a randomizing field name, but are also filling out the honeypot and getting caught.

  3. Some seem to be coping with the honeypot just fine, and either getting caught by stopforumspam or getting through. I wonder if these could be people spamming rather than bots, or if they are just smart enough bots that can avoid the honeypot.

Observations

  • By far the most attempts fall into category 1. Quite a lot of these also set the timezone to -12 for some reason. Some of these are also using www.fluxbb.org rather than fluxbb.org for some reason, and 1 even used bg.fluxbb.org (??).

  • One of the registrations in category 2 actually had the referer "http://fluxbb.org/forums/register.php?action=register", which implies the form was submit after receiving an error and correcting it. This seems rather odd.

Offline

#36 2010-05-06 19:11:01

MattF
Member
From: South Yorkshire, England
Registered: 2008-05-06
Posts: 1,233
Website

Re: Anti Spam in core

Just on a tangent in methodology, has anyone tried using a prefilled input and just telling the registering user to delete all text from that input, then check to make sure the input has no content on submission? If bots have a liking for filling input fields, working arse ways round might work?


Screw the chavs and God save the Queen!

Offline

#37 2010-05-06 19:47:04

Franz
Lead developer
From: Germany
Registered: 2008-05-13
Posts: 6,715
Website

Re: Anti Spam in core

Smartys wrote:

2. Adding something like that would require keeping track of "state" in some form (ie: sessions) which FluxBB currently does not do.

Wouldn't a timestamp as a hidden field be enough???


fluxbb.de | develoPHP

"As code is more often read than written it's really important to write clean code."

Offline

#38 2010-05-06 20:20:51

FSX
Former Developer
From: NL
Registered: 2008-05-09
Posts: 818
Website

Re: Anti Spam in core

Franz wrote:
Smartys wrote:

2. Adding something like that would require keeping track of "state" in some form (ie: sessions) which FluxBB currently does not do.

Wouldn't a timestamp as a hidden field be enough???

It's possible to change the value of that field.

Offline

#39 2010-05-06 20:27:44

Reines
Administrator
From: Scotland
Registered: 2008-05-11
Posts: 3,197
Website

Re: Anti Spam in core

I was going to say that, but then if a bad was to be specifically made to get around that, it could also be made to just wait 30 seconds before submitting the data.

Offline

#40 2010-05-06 20:49:08

Franz
Lead developer
From: Germany
Registered: 2008-05-13
Posts: 6,715
Website

Re: Anti Spam in core

Exactly. There is no method that doesn't annoy users and can't be adapted by a mod.


fluxbb.de | develoPHP

"As code is more often read than written it's really important to write clean code."

Offline

#41 2010-05-06 20:53:50

FSX
Former Developer
From: NL
Registered: 2008-05-09
Posts: 818
Website

Re: Anti Spam in core

Maybe sending a token with the form (anti-CSRF) also helps. It at least blocks out post requests from external hosts.

Offline

#42 2010-05-06 21:26:27

damaxxed
Member
From: Germany
Registered: 2008-05-16
Posts: 353

Re: Anti Spam in core

OK, let's sum this up.

There is no general, static solution that can be applied to any board. Bots can be rewritten to fit every anti-bot-test (sooner or later). Additionally, every anti-spam measure must not interfere with the accessibility. This includes that solutions shouldn't depend on CSS, Flash or JavaScript in order to enable basic mobile browsers, text-only web browsers to use the registration form.

You have to ask yourself which visitors may be excluded in order to use anti-spam systems.

One that is easily adaptable and in my opinion a very good idea, is the question-answer captcha. Every forum admin can specify one or more questions which will be asked on registration. This eliminates a general spam bot for all FluxBB forums.

The hidden fields with css measure may look promising to me, but I personally don't know many factors: How easy is it for bots to parse and apply CSS and JavaScript? How do screenreaders, text-only browsers and old mobile devices handle this?

Image CAPTCHA implementations mustn't be disregarded. Even if most are cracked and every will be cracked sooner or later, the reCAPTCHA looks really good. It doesn't require JS and has audio files as alternatives for blind users.

Offline

#43 2010-05-06 22:23:44

MattF
Member
From: South Yorkshire, England
Registered: 2008-05-06
Posts: 1,233
Website

Re: Anti Spam in core

I hate recaptcha and I have practically perfect eyesight. The way they obfuscate the text makes it nigh on impossible to read, half the time, IMHO.


Screw the chavs and God save the Queen!

Offline

#44 2010-05-07 09:19:37

Reines
Administrator
From: Scotland
Registered: 2008-05-11
Posts: 3,197
Website

Re: Anti Spam in core

damaxxed wrote:

There is no general, static solution that can be applied to any board. Bots can be rewritten to fit every anti-bot-test (sooner or later).

I think the important point in this is, it is easy to protect your own board using passive techniques (i.e. honeypot etc.), but if included as part of the core it would only be a short term solution until bots adapted to handle it.

StopForumSpam seems to work to some extent, it won't remove all spam but it will cut the amount down. However I would be reluctant to code reliance on a 3rd party service into the core. A better option is probably to have it available as an official modification.

damaxxed wrote:

The hidden fields with css measure may look promising to me, but I personally don't know many factors: How easy is it for bots to parse and apply CSS and JavaScript? How do screenreaders, text-only browsers and old mobile devices handle this?

As above, this isn't really a viable solution for the core, but again it would make a good mod. Using display:none with CSS is okay for screenreaders, but doesn't work in text-based browsers. What I've done here is used display:none and a label that says "If you are human, leave this blank!", so if CSS isn't enabled the person knows not to fill it in. Using HTML comments to make it invisible should work fine for all kinds of browsers, but would be easier for a bot to cope with too.

damaxxed wrote:

Image CAPTCHA implementations mustn't be disregarded. Even if most are cracked and every will be cracked sooner or later, the reCAPTCHA looks really good. It doesn't require JS and has audio files as alternatives for blind users.

This would also make a great mod, but personally I hate having CAPTCHAs and I think it looks incredibly ugly. If the choice was CAPTCHA or spam I'd obviously choose the CAPTCHA, but between CAPTCHA and customized honeypot, I'd rather the honeypot.

My feeling is our best solution is what we've said all along, provide lots of different options in the form of mods and let users (by users I mean forum admins) decide which is best for their situation. To help less technical users maybe the best idea would be if we made some official mods for each of the techniques (honeypot/blacklist/CAPTCHA) which are as easy to install as possible, and link to them from the download page.

Offline

#45 2010-05-07 09:44:57

Paul
Developer
From: Wales, UK
Registered: 2008-04-27
Posts: 1,653

Re: Anti Spam in core

Its always possible that once this and other boards have been live for a while a solution will present itself or at least it will become clearer what works and what doesn't in which case it could be implemented in a service release.


The only thing worse than finding a bug is knowing I created it in the first place.

Offline

#46 2010-05-07 09:46:27

Reines
Administrator
From: Scotland
Registered: 2008-05-11
Posts: 3,197
Website

Re: Anti Spam in core

Some results from last night...

  • 186 spammers caught by honeypot. Looking at the access logs you can easily tell these are bots. The same IP uses a random set of user_agents and all requests are either a forum/post, the login page, or register page. Not a single other file (i.e. CSS/images) is requested.

  • 7 spammers caught by StopForumSpam. I'm sure this number would be higher if we didn't have the honeypot, but since most are filtered out by that they never get checked. What's interesting to note is looking at the access logs, these appear to be legitimate people. They avoid the honeypot just fine, and their requests include CSS/images as they should. Their user_agent is also consistent (per IP/user) as you would expect. They also browse from login, to index, to profile, to signature section of profile, following the links as a human would. Either they are incredibly smart bots, or human generated spam.

  • 13 registrations allowed. I think the majority of these actually should fall into the above category, but weren't in the StopForumSpam database.

Offline

#47 2010-05-07 10:06:41

Paul
Developer
From: Wales, UK
Registered: 2008-04-27
Posts: 1,653

Re: Anti Spam in core

The point is that if the number of spam registrations is small enough then its possible to manually delete fake users if its done regularly.  With 200 a day it just wasn't practical.


The only thing worse than finding a bug is knowing I created it in the first place.

Offline

#48 2010-05-07 10:09:55

Reines
Administrator
From: Scotland
Registered: 2008-05-11
Posts: 3,197
Website

Re: Anti Spam in core

Sure, well assuming no-one makes a new bot to avoid the honeypot any time soon, we're down from a potential 206 last night to just 13. With the inclusion of Akismet and/or just plain blocking of URLs for people under X posts the human spammers may give up as well.

Offline

#49 2010-05-07 20:45:58

Jérémie
Member
From: France
Registered: 2008-04-30
Posts: 629
Website

Re: Anti Spam in core

Would these honeypots, and an optionnal Akismet linkage, be included in the core in RC4?

SPAM affect everyone, even admins not having any PHP clues.

Last edited by Jérémie (2010-05-07 21:01:04)

Offline

#50 2010-05-07 21:54:54

Franz
Lead developer
From: Germany
Registered: 2008-05-13
Posts: 6,715
Website

Re: Anti Spam in core

As stated above, that wouldn't make sense, because not every anti-spam solution works for (and against) everybody. It's only a combination or a careful selection that is appropriate.

Also: RC4? Please, not.


fluxbb.de | develoPHP

"As code is more often read than written it's really important to write clean code."

Offline

Board footer

Powered by FluxBB