Photograph

« Asking the Right Questions | Back to Home | Google's (Changing) Philosophy »

July 17, 2005  Angelina Jolie Nude

Angelina JolieComment and Trackback spam on my website has increased over the last couple months and is driving me crazy. Everything from promises of nude celebrity photographs to poker to subjects that are not family oriented were being advertised on my site. The time came to do something about it. Here is my long, strange trip down the road of preventing spam:

Renaming MT-Comments.cgi

I started where everyone starts. Renaming MT-comments.cgi so that machines searching the web for that file have a harder time finding it. I hit a bug and reverted back. Spammers 1, Johnnie 0.

MT-Blacklist

Most people using Movable Type install MT-Blacklist, which blocks the IP addresses associated with spammers. I installed MT-Blacklist 2.0 but it was incompatible with Berkeley DB, my database. An older version (1.6.5) is compatible with Berkeley DB but not with Movable Type 3.1.x. Sigh. Spammers 2, Johnnie 0.

MT-DisguiseTrackbackURL

My next step was to install MT-DisguiseTrackbackURL. This takes the Trackback URL and uses JavaScript to render it on the page. This way a spambot will need to run JavaScript in order to learn about the URL. This isn't a very robust way to protect the Trackback URL, but it was easy to install, and I needed a quick victory. Spammers 2, Johnnie 1.

MT-Scode

Next I tried to installed Mt-Scode. This allows you to put a graphic with numbers and letters next to your comment submission form, and require people to write the value they see in an edit box. This keeps the spambots from being able to leave a comment. Unfortunately this required a graphics library called GD. I downloaded GD, but to install it I needed a C complier like GCC or CC, which I couldn't find when I SSH-ed to my account. Not looking good... Spammers 3, Johnnie 1.

MT-Bayesian

Then I decided to install MT-Bayesian. This uses a training algorithm to learn about the properties of spam and then hide those comments from showing up. The original MT-Bayesian is incompatible with Movable Type 3.1.x but I found an updated one. Unfortunately it was extremely processor intensive (read: painfully slow). It also did not rebuild templates in an intelligent way, requiring me to rebuild all individual posts even if the change was only to one. Training was taking a really long time, and due to some bug the comments were still showing up. I thought it was hopeless at this point: Spammers 4, Johnnie 1.

Asking a Question

Then I read about a simple idea: just ask a question and have the person enter the result in the form. You'll notice the question I ask in the comments area. This makes it harder for a spambot to leave comments.

The code for this was really simple. I added the input field to my Individual Entry templates with the name/id of X, and added this snipet of code to MT-Comments.cgi, right under the line that says 'use strict;':

use CGI qw(:standard);

my $data = param('X');
die unless (($data eq 'the-answer') || ($data eq 'The-answer'));

Spammers 4, Johnnie 2.

Movable Type 3.2

Movable Type 3.2 is currently in Beta, but it looks like it has MT-Bayesian type functionality built in. Hopefully the combination of the work I've done so far and Movable Type 3.2 will help me even the score.

Posted by johnnie at July 17, 2005 11:55 PM

Comments

Testing my comment question field.

Posted by: Johnnie Manzari at July 18, 2005 12:14 AM


I am using MT Approval myself, and have not had one comment spam since. It basically *requires* the commenter to preview the comment before posting. It's a bit crude, but it works, and it's just one more mouse click instead of answering a question or typing in 34324234...

Trackback spam, on the other hand...

Posted by: Jonathan at July 18, 2005 01:23 AM


Ah, my link to MT Approval didn't show up as html I guess isn't allowed:

http://jayseae.cxliv.org/approval/

You can see it in action at my site if you are interested -

http://www.mymoneyblog.com

Posted by: Jonathan at July 18, 2005 01:25 AM


Johnnie,

Congrats on having the best "click here now!" link title on the 9rules homepage. :)

Posted by: Matthew Oliphant at July 18, 2005 05:17 AM


Yeah, damn you Johnnie. I clicked on that link quick! You are now obligated to provide pics.

Posted by: Scrivs at July 18, 2005 09:08 AM


My #1 comment spam was Angelina Jolie Nude, so I thought it would be a fitting title. Scrivs is right, though. I should probably dig up a picture.

Posted by: Johnnie Manzari at July 18, 2005 09:50 AM


I actually dropped my squirrel monkey trick eventually in favour of MT-Keystrokes (see more about it http://www.sevitz.com/site/2005/03/new_host_new_niggles.asp)

Which cured 95% of my spam. All spam I now get is by someone who has to type it in. It's pretty much the same as the question thing, but without the question.

Going to have a look at MTDisguiseTrackbackURL though as I am getting loads of TB spam, which stops me getting normal trackbacks from nice people like you.

Posted by: Adrian at July 18, 2005 02:17 PM


This sucks how I can post under any name I want =T

[Editor: This was posted anonymously, but using the name Johnnie Manzari.]

Posted by: Unknown at July 19, 2005 02:00 AM


Plus that title is likely to atract more spam or at least Google traffic from people not so interested on the topics you normally cover.

Posted by: Julio Alonso at July 19, 2005 05:03 AM


I've been having similar problems with my blog, made all the more interesting to me due to the fact that I wrote my own little system. My setup is purposefully very simple, and my goal has been to allow anyone to comment as easily as is possible.

Maybe seven or eight months ago, comment spam started appearing. My first step was to rename the username and comment fields to more random words, hoping to confuse any spiders searching for comment fields. This actually worked for a while.

When I again started getting comment spam, it was attached to just a few of my posts, and all older ones. The next step was to make it possible to "turn off" comments on older posts. This was just a flag in the database, and all it did was stop my website from drawing up the comment forms when you went to a marked post. I knew this wouldn't work if they had scripts just posting on their own, basically having memorized my forms, and not actually going through them. The spam didn't really stop, so I'm guessing that this is actually the case.

My next step, which I've started to code, but haven't tested and implemented yet is to basically add a obfuscated timestamp that gets posted as a hidden value in the form along with a comment submitted from my blog. Then the server will reverse the stuff I did to the timestamp, compare that result to a current timestamp, and if it's been less than 10 minutes, it'll accept the comment. So even if these spam scripts store all the values that my commenting system requires, this one will change after a short period of time, and their later requests will be ignored. All without requiring any extra work on the part of the commenter.

Does that make sense? Will it work they way I want it to?

Posted by: shawn at July 22, 2005 09:34 AM


BUY VAIGRA HERE!!

...just kidding :D

Posted by: Joel Bernstein at July 22, 2005 09:54 AM


Shawn, let me know if the crazy timestamp thing works. I hadn't heard of anyone trying something like that. Things are a lot better for me now. I still get some trackback spam, but it's gone from nightmarish to only slightly annoying.

Posted by: Johnnie Manzari at July 22, 2005 03:50 PM


After laboring with MT-Blacklist for a few months, I've now moved across the SpamLookup[1] with fantastic success. No spam, nada.

Have you tried it?

[1] http://bradchoate.com/projects/spamlookup/

Posted by: chris at July 22, 2005 06:56 PM


I get an internal server error when a person enters a wrong answer. Is that the intended result or have I done something wrong when implementing "Asking a Question"?

Posted by: Clark MacLeod at July 26, 2005 10:59 AM


Listen to Chris... SpamLookup is the only thing that saved me from migrating to WordPress months ago. Of course, after wasting three hours of my life backing up, unsuccessfully installing 3.2, and then reverting back, I'll still be making that switch... And soon!

Posted by: Aurorealis at July 26, 2005 07:10 PM


The internal server error is what happens now because I'm just killing the process. It's pretty ugly but I didn't have time to set it up so that it redirects to an error page. Per the recommendations here I'm going to look in to SpamLookup because the ask-a-question thing is not a great user experience, and it sounds like moving to 3.2 is not as smooth as MT has advertised.

Posted by: Johnnie Manzari at July 27, 2005 11:40 AM


Great blog, thank you!

Posted by: andrey at January 11, 2006 09:53 AM


Good blog

Serg

Posted by: Serg at June 6, 2006 03:34 AM


HI,

Johnne So lucky to join in this group, I love Your picture (especially) ANGELINA, Pls sent me on this e.mail ID

Thank You

Posted by: Mahe at June 29, 2006 03:42 AM


Post a Comment




Remember Me?


Type the word 'Manzari' in to the box below:
(This is to limit spam)

the 9rules Network logo