Honeypot + StopForumSpam Mod 1.0.1

Koos · 2012-11-05 22:48:35

##
##
##        Mod title:  Honeypot + StopForumSpam Mod
##
##      Mod version:  1.0.1
##  Works on FluxBB:  1.4.8, 1.4.9, 1.5.0, 1.5.1
##     Release date:  2012-11-16
##           Author:  Koos (pampoen10@yahoo.com)
##Based on patch by:  Reines (jamie@jamierf.co.uk)
##
##      Description:  Spam Prevention Mod
##
##   Affected files:  header.php
##                    profile.php
##                    register.php
##                    lang/English/profile.php
##                    lang/English/register.php
##
##       Affects DB:  New table:
##                       'test_registrations'
##                    New options in 'config' table:
##                       'o_stopforumspam_check'
##                       'o_stopforumspam_api'
##
##            Notes:  You can skip steps 7-21 and 31-33 if you do not need the
##                    ability to manually report spammers to the StopForumSpam
##                    database. Following these steps will add an extra option for
##                    admin when deleting users called "Delete user & report spam".
##
##       DISCLAIMER:  Please note that "mods" are not officially supported by
##                    PunBB. Installation of this modification is done at your
##                    own risk. Backup your forum database and any and all
##                    applicable files before proceeding.
##
##

I was looking for a spam solution for PunRes and discovered that the FluxBB forum uses a Honeypot + StopForumSpam combo. Reines was friendly enough to share the patch to add this method of spam protection here. This mod is based on this patch. All I added was options to enable/disable certain features, and also some stats in the mod's admin cp.

Since installing it there has only been about 2 or 3 spam registrations. About 100-200 spam registration attempts are blocked per day. The Honeypot blocks almost 100% of the bots. And the StopForumSpam as Reines mentioned serves as a second barrier against human spammers.

The greatest thing about this mod is that it's completely invisible to normal users!

Here is an explanation by Reines on how it works:

Reines wrote:

We have the username field renamed on the register form, and a new hidden field added with the old name. When the form is submit we check if the hidden field has been filled out, if it has then the user is a bot. This seems to catch 100% of bots. Some of the bots target stock PunBB/FluxBB installs and hence fill out the field with the old name and leave the new field blank, hence the registration is denied as the username field is blank. Some other bots fill out all fields, hence the registration is denied because they filled out the hidden field.
If they pass those checks, we check the IP and email address (not username) against the stopforumspam API. This catches a few human spammers.

Some additional info:

http://fluxbb.org/forums/viewtopic.php?pid=43297#p43297
http://fluxbb.org/forums/viewtopic.php?pid=34372#p34372
http://fluxbb.org/forums/viewtopic.php?id=5263
http://fluxbb.org/forums/viewtopic.php?pid=31641#p31641

Some screenshots of the mod's admin cp:

Changelog
v1.0.1

When a StopForumSpam API key is provided, blocked registration attempts are reported to the StopForumSpam database. Added an additional check to see if the usernamefield is indeed hidden. If not, blocked registration attempts will not be reported. This is to prevent invalid reports being sent to the StopForumSpam database in case the user did not install the mod correctly.
Now also compatible with PostreSQL and SQLite
Added localization support to mod's admin cp
Fixed some bugs in mod's admin cp

v1.0

Initial release

Download Honeypot + StopForumSpam Mod

Last edited by Koos (2012-11-17 13:28:19)

sklerder · 2012-11-06 17:42:16

Hi !

Great
Many thanks, Koos !

But I feel there may be a little error :
The original field, on the form, for the user name, is "req_user".
And after modification, it is renamed "req_username" ...

I think the following lines in your "readme.txt" should be modified to keep the original names.

Line 194 :
You : req_username = empty($username) ? pun_trim($_POST['req_username']) : $username;
Me : req_username = empty($username) ? pun_trim($_POST['req_user']) : $username;
Line195 :
You : if (!empty($_POST['req_username']))
Me : if (!empty($_POST['req_user']))
Line 237 :
You : <label class="required usernamefield"><strong><?php echo $lang_register['If human'] ?></strong><br /><input type="text" name="req_username" value="" size="25" maxlength="25" /><br /></label>
Me : <label class="required usernamefield"><strong><?php echo $lang_register['If human'] ?></strong><br /><input type="text" name="req_user" value="" size="25" maxlength="25" /><br /></label>

Last edited by sklerder (2012-11-06 17:43:10)

Koos · 2012-11-06 18:50:45

I'm sure there must be a reason why Reines chose to use req_username. Maybe it's more effective against bots. The FluxBB support forum also uses it as req_username, and it seems to work well for them. So for now I will leave it as is.

sklerder · 2012-11-06 18:59:28

I thought the reason was the porting from PunBB to FluxBB (with PunBB, this field is effectively named "req_username").

But bots probably try the two names, thats why it still works.

Maybe with the "original" name, more bots could be "caught"

quy · 2012-11-06 19:14:28

Please consider adding evidence info (URL, signature, and last posting if any) when reporting. Thanks.

Koos · 2012-11-06 19:44:15

sklerder wrote:

Maybe with the "original" name, more bots could be "caught"

In it's current format it seems to catch almost all bots. So I think I'll stick with this tried and tested format for now

quy wrote:

Please consider adding evidence info (URL, signature, and last posting if any) when reporting. Thanks.

Thanks for the hint, I might consider it for future versions. But at the moment evidence info is not being used by the StopForumSpam service, as mentioned on their add page: "The evidence field will be used at a later date to provide a URL scanning service."

sklerder · 2012-11-06 20:10:57

Koos wrote:

In it's current format it seems to catch almost all bots. So I think I'll stick with this tried and tested format for now

And I already use this honeypot with "req_user" (but on a French forum, may change some things ...), and it works as well

Koos wrote:

quy wrote:
Please consider adding evidence info (URL, signature, and last posting if any) when reporting. Thanks.
Thanks for the hint, I might consider it for future versions. But at the moment evidence info is not being used by the StopForumSpam service, as mentioned on their add page: "The evidence field will be used at a later date to provide a URL scanning service."

As Koos said, evidence is not mandatory

I provide this evidence info when possible (for example, "Automated registration detected."), but to report an URL or signature on registration, this is clearly not possible, these informations can't have been set at this moment

Spiky · 2012-11-07 11:53:02

Thanks for this mod.
I already use modification "StopForumSpam" proposed by adaur on fluxbb.fr without Honeypot which works fine. Your change is more full and I have intention to install it.

Small note: why you have not created a language file instead of having hard-coded into the plugin?

adaur · 2012-11-07 17:59:38

That looks good Koos, thanks for this great mod .

However,

Spiky wrote:

why you have not created a language file instead of having hard-coded into the plugin?

I can't agree more; I wanted to make the french translation but without a localized plugin file, it's a real pain.

I think you should make some changes in the plugin:

echo "\t\t\t\t\t\t".'<tr><td class="tcl"><a href="profile.php?id='.$cur_user['id'].'">'.pun_htmlspecialchars($cur_user['username']).'</a></td><td class="tc2">'.$cur_user['email'].'</td><td class="tc3">'.$cur_user['num_posts'].'</td><td class="tc4">'.$cur_user['url'].'</td><td class="tc5">'.$cur_user['signature'].'</td><td class="tcr">'.format_time($cur_user['registered'], true).'</td></tr>'."\n";

$cur_user['num_posts'] : apply forum_number_format
$cur_user['url'] : possible XSS injection; apply pun_htmlspecialchars
$cur_user['signature'] : you should parse it like in viewtopic (I think it's vulnerable to XSS for now)

		if (isset($signature_cache[$cur_post['poster_id']]))
			$signature = $signature_cache[$cur_post['poster_id']];
		else
		{
			$signature = parse_signature($cur_post['signature']);
			$signature_cache[$cur_post['poster_id']] = $signature;
		}

Last edited by adaur (2012-11-07 18:00:34)

Koos · 2012-11-07 21:46:50

Thanks Spiky and adaur for the suggestions. I will try to update the mod tomorrow.

053+ · 2012-11-08 10:37:59

After update = This mod will be great!

Franz · 2012-11-09 10:14:37

@Koos:

I'm sorry, but I had to delete this modification for now (made a backup). Please contact pedigree to sort this out.

Koos · 2012-11-09 18:26:17

I see that the StopForumSpam terms page says:

"without validation, is strictly prohibited"

"Only validated spam will be tolerated"

Is it not enough validation that a field is filled in which only bots can see?

pedigree mentioned that this mod "submits non-validated data". As I mentioned above, this mod is entirely based on the patch Reines shared here, and is also used here on the FluxBB support forum. It helped FluxBB attain the 6th spot on the list of StopForumSpam contributors with 275,808 entries. Should these entries thus be invalidated?

pedigree · 2012-11-10 19:01:11

If you would like to setup a honeypot, then please contact me and I will go out of my way to help out. I just need to know if you have a honeypot submitting, so that I can manage anything that comes in. At the moment, I get a very large number of removal requests from all types of sites and I'm just trying to get everything under control before it either gets too much and I say "screw it" and turn off api keys or the site. I really do not want to do the later but without some form of control, then I cannot get on top of things, so that I can then start with new work. I'll start an email discussion with Reines/Franz and Koos over this weekend so that things can be sorted out. In the meantime, I know that flux is running this, so there is no need to remove it.

sklerder · 2012-11-10 22:40:51

Hi !

I feel concerned by this answer, pedigree, and to be honest, was waiting your answer

I have recently set up a forum, and implemented multiple levels of detection based on the works from Reines, Koos, adaur and blissend.

What I do at registration phase :
1) Detect by "HoneyPot method" (legacy field hidden to humans - except screen readers - and a new field to submit the username).
If an automated registration is detected, I submit it as a spammer to SFS, with an evidence text, and end it with a registration deny (as Koos, I think that the facts the required field is empty and the legacy field not empty are enough as evidence of future spamming, even if it could be seen as an "obstacle to the presumption of innocence").
2) If no previous detection, request to SFS (IP and email).
In a previous version, I submitted (again) the spammers to SFS, but no more now
3) If no answer from SFS (it happends from times to times), request to DNSBL (sbl.spamhaus.org, xbl.spamhaus.org, b.barracudacentral.org, opm.tornevall.org, in this order)

At login phase (due to an old history on another forum), I also verify with SFS (DNSBL if SFS does'nt answer) to avoid some bad guys.

I know that I should verify with DNSBL before SFS, but I had too much false positives on DNSBL (too much latency on blacklists, particularly to remove; more, on tornevall.org, my server's IP is blacklisted, and I can't remove it, for the moment ) ...

And I know that the way I try to fight the SPAM (and spammers) could (and should) be improved, but it will be difficult to do without your collaboration.

Spamming is becoming a plague on forums, and the possibility to submit spammers to a centralized database as SFS is one of the best way to share our works and experiences (and reading the name "StopForumSpam", it sounds to me it's the appropriate service to do it ).

So your advices and help, pedigree, would be welcome to do it the best way for all.
If you see things to improve the way I do it, I'm ready to listen and try them, whenever possible.

Thanks in advance

P.S. : Please excude my bad English, I'm French ...

Last edited by sklerder (2012-11-11 23:38:35)

pedigree · 2012-11-14 20:31:43

I've talked to Koos about this and have sorted it all out. If you follow the code instructions carefully, then everything will be alright and you wont get banned from StopForumSpam.

ryotiger · 2012-11-14 20:48:40

So there is a new version of this mod ?

Koos · 2012-11-16 21:18:12

Honeypot + StopForumSpam Mod 1.0.1 now available for download.

Changes made:

When a StopForumSpam API key is provided, blocked registration attempts are reported to the StopForumSpam database. Added an additional check to see if the usernamefield is indeed hidden. If not, blocked registration attempts will not be reported. This is to prevent invalid reports being sent to the StopForumSpam database in case the user did not install the mod correctly.
Now also compatible with PostreSQL and SQLite
Added localization support to mod's admin cp
Fixed some bugs in mod's admin cp

Franz · 2012-11-16 21:23:23

Thanks, Koos! I can only recommend this!

sklerder · 2012-11-16 22:01:50

Hi !

Many thanks, Koos

Here is the French translation (file lang/French/honeypot_sfs_plugin.php) :

<?php

// Language definitions used by the Honeypot + SFS mod
$lang_honeypot_sfs_plugin = array(

'Description'					=>	'Ce plugin est utilisé pour contrôler les réglages pour le mod Honeypot + StopForumSpam.',
'Options'						=>	'Options',
'Settings'						=>	'Réglages',
'StopForumSpam check label'		=>	'Contrôle StopForumSpam',
'StopForumSpam check help'		=>	'Si l\'utilisateur qui tente de s\'inscrire passe le contôle du "honeypot", vérifier l\'adresse IP de l\'utilisateur et son adresse mail (mais pas le pseudo) auprès de la base de données de "blacklistage" de StopForumSpam. Bien que le "honeypot" traite presque 100% des robots, le service StopForumSpam est utilisé comme seconde barrière contre les spammeurs humains.',
'StopForumSpam API label'		=>	'API StopForumSpam',
'StopForumSpam API help'		=>	'Votre clé d\'accès à l\'API StopForumSpam. Si laissé vide, les tentatives d\'inscription "spammeuses" ne seront pas enregistrées auprès du service blacklistage de StopForumSpam.',
'Options updated redirect'		=>	'Options mises à jour. Redirection …',

// Search users feature
'Search users head'				=>	'Recherche d\'utilisateurs',
'Search users info'				=>	'Cette fonctionnalité vous permet de rechercher les utilisateurs dont la signature comporte un lien mais n\'ayant jamais posté. Ceci dans le but de rechercher les spammeurs qui ont réussi à passer outre les contrôles "HoneyPot" et StopForumSpam. Les résultats de recherche sont limités au 50 derniers  utilisateurs inscrits correspondant à ces critères.',

// Statistics
'Statistics'					=>	'Statistiques',
'Collecting stats since label'	=>	'Récolement des statistiques depuis',
'Num days'						=>	'%s jours',
'Not available'					=>	'Non disponible',
'Total label'					=>	'Total',
'Average last 7 days label'		=>	'Moyenne des 7 derniers jours',
'Maximum day label'				=>	'Jour maximum',
'Blocked last 14 days label'	=>	'Bloqués les derniers 14 jours',

'Not spam info'					=>	'Non SPAM : %s',
'Blocked by Honeypot info'		=>	'Bloqués par Honeypot : %s',
'Blocked by SFS info'			=>	'Bloqués par SFS : %s',
'per day'						=>	'par jour',
'Date'							=>	'Date',
'Total'							=>	'Total',

);

And I'll try later to merge with the plugin I've done (adding DNSBL) ...

[Edit]
Some simple quotes were missing
[/Edit]

Last edited by sklerder (2012-11-17 11:41:58)

Squiggles · 2012-12-27 03:18:00

I have installed this mod but a user who's IP was listed in the StopForumSpam database was still able to register. Why is that?

Does the IP and Email Address have to both exist in the SFS database for the user to be denied registration?

Also when I originally tested the mod (registered a non spam user), the Non Spam statistic counter changed by increment of 1, this new user did not trigger any change in the statistic counter. Why is that?

Edit: My test account shows up under test_registrations in my database.

Last edited by Squiggles (2012-12-27 03:46:59)

Squiggles · 2012-12-28 04:13:06

It seems to be working now, I'm not sure what was going on

orkneywd · 2013-01-09 01:06:37

The readme file says you need to make changes to header.php but there are none listed in the readme? I tried the mod and it didn't work for me (kept triggering the Javascript "Username is a required field" alert). I looked in header.php and it looks like the code has changed at some point in time?

I'm on 1.5.0 and have the following:-

<script type="text/javascript">
/* <![CDATA[ */
function process_form(the_form)
{
	var required_fields = {
<?php
	// Output a JavaScript object with localised field names
	$tpl_temp = count($required_fields);
	foreach ($required_fields as $elem_orig => $elem_trans)
	{
		echo "\t\t\"".$elem_orig.'": "'.addslashes(str_replace('&#160;', ' ', $elem_trans));
		if (--$tpl_temp) echo "\",\n";
		else echo "\"\n\t};\n";
	}
?>
	if (document.all || document.getElementById)
	{
		for (var i = 0; i < the_form.length; ++i)
		{
			var elem = the_form.elements[i];
			if (elem.name && required_fields[elem.name] && !elem.value && elem.type && (/^(?:text(?:area)?|password|file)$/i.test(elem.type)))
			{
				alert('"' + required_fields[elem.name] + '" <?php echo $lang_common['required field'] ?>');
				elem.focus();
				return false;
			}
		}
	}
	return true;
}
/* ]]> */
</script>

I changed it to some old header code from the 1.4 branch, and added in code out of Reines original patch and all is now working

<script type="text/javascript">
/* <![CDATA[ */
function process_form(the_form)
{
	var element_names = new Object()
<?php

	// Output a JavaScript array with localised field names
	foreach ($required_fields as $elem_orig => $elem_trans)
		echo "\t".'element_names["'.$elem_orig.'"] = "'.addslashes(str_replace('&#160;', ' ', $elem_trans)).'"'."\n";

?>

	if (document.all || document.getElementById)
	{
		for (var i = 0; i < the_form.length; ++i)
		{
			var elem = the_form.elements[i]
			if (elem.name && elem.name != "req_username" && elem.name.substring(0, 4) == "req_")
			{
				if (elem.type && (elem.type=="text" || elem.type=="textarea" || elem.type=="password" || elem.type=="file") && elem.value=='')
				{
					alert("\"" + element_names[elem.name] + "\" <?php echo $lang_common['required field'] ?>")
					elem.focus()
					return false
				}
			}
		}
	}

	return true
}
/* ]]> */
</script>

Thanks for the great mod, hope it works

Studio384 · 2013-02-01 14:55:52

As seen in round up #1 - Merry Christmas and a Happy New Year

Ememone · 2013-02-03 14:12:31

is it working with FluxBB 1.5.2? Will be great...

Forums

#1 2012-11-05 22:48:35

Honeypot + StopForumSpam Mod 1.0.1

#2 2012-11-06 17:42:16

Re: Honeypot + StopForumSpam Mod 1.0.1

#3 2012-11-06 18:50:45

Re: Honeypot + StopForumSpam Mod 1.0.1

#4 2012-11-06 18:59:28

Re: Honeypot + StopForumSpam Mod 1.0.1

#5 2012-11-06 19:14:28

Re: Honeypot + StopForumSpam Mod 1.0.1

#6 2012-11-06 19:44:15

Re: Honeypot + StopForumSpam Mod 1.0.1

#7 2012-11-06 20:10:57

Re: Honeypot + StopForumSpam Mod 1.0.1

#8 2012-11-07 11:53:02

Re: Honeypot + StopForumSpam Mod 1.0.1

#9 2012-11-07 17:59:38

Re: Honeypot + StopForumSpam Mod 1.0.1

#10 2012-11-07 21:46:50

Re: Honeypot + StopForumSpam Mod 1.0.1

#11 2012-11-08 10:37:59

Re: Honeypot + StopForumSpam Mod 1.0.1

#12 2012-11-09 10:14:37

Re: Honeypot + StopForumSpam Mod 1.0.1

#13 2012-11-09 18:26:17

Re: Honeypot + StopForumSpam Mod 1.0.1

#14 2012-11-10 19:01:11

Re: Honeypot + StopForumSpam Mod 1.0.1

#15 2012-11-10 22:40:51

Re: Honeypot + StopForumSpam Mod 1.0.1

#16 2012-11-14 20:31:43

Re: Honeypot + StopForumSpam Mod 1.0.1

#17 2012-11-14 20:48:40

Re: Honeypot + StopForumSpam Mod 1.0.1

#18 2012-11-16 21:18:12

Re: Honeypot + StopForumSpam Mod 1.0.1

#19 2012-11-16 21:23:23

Re: Honeypot + StopForumSpam Mod 1.0.1

#20 2012-11-16 22:01:50

Re: Honeypot + StopForumSpam Mod 1.0.1

#21 2012-12-27 03:18:00

Re: Honeypot + StopForumSpam Mod 1.0.1

#22 2012-12-28 04:13:06

Re: Honeypot + StopForumSpam Mod 1.0.1

#23 2013-01-09 01:06:37

Re: Honeypot + StopForumSpam Mod 1.0.1

#24 2013-02-01 14:55:52

Re: Honeypot + StopForumSpam Mod 1.0.1

#25 2013-02-03 14:12:31

Re: Honeypot + StopForumSpam Mod 1.0.1

Board footer