Spam: The Final Solution

tags:

Until the mid 90s, spam was a non-issue. It was exciting to get email. The web was also virtually spam-free. Netizens respected one another and everything was very pleasant. Spam Those days are long gone. Fortunately, there are some pretty amazing tools out there for fighting email spam. I use a combination of SpamAssassin on the server side and Thunderbird (with its wonderful built in junkmail filters) on the desktop. I am sent thousands of spam messages a day that I never see thanks to these tools.

But approximately five years ago, a new type of spam emerged which exploited not email but the web. Among this new wave of abuse, my personal favorite, comment spam.

I love getting comments on my blog. I also like reading comments on other blogs. However, it's not practical to simply allow anyone who wants to leave a comment, as within a very short period of time, blog comments will be overrun with spam generated by scripts that exploit sites with permissive comment privileges. To prevent this, most sites require that you log in to post a comment. But this may be too much to ask of someone who just wants to post a quick comment as they pass through. I often come across blog postings which I would like to contribute to, but I simply don't bother because the site requires me to create an account (which I'd likely only use once) before posting a comment. Not worth it. Another common practice is the use of "captchas" which require a user enter some bit of information to prove they are human and not a script. This works fairly well, however, it is still a hurdle that must be jumped before a user can post a comment. And as I've personally learned, captchas, particularly those that are image based, are prone to problems which may leave users unable to post a comment at all.

As email spam grew, there were various efforts to implement similar types of protection, requiring by the sender to somehow verify he was not a spammer (typically by resending the email with some special text in the subject line). None of these solutions are around anymore because they were just plain annoying. SpamAssassin and other similar tools are now used on most mail servers. Savvy email users will typically have some sort of junkmail filter built into their email client or perhaps as part of an anti-virus package. And spam is much less a nuisance as a result.

What we need for comment spam is a similar solution. One that works without getting in the way of the commenter or causing a lot of work for the blog owner. Turn it on, and it works. I've recently come across just such a solution for blogs which also happens to have a very nice Drupal module so you can quickly and easily put this solution to work on your own Drupal site.

Enter Akismet

It's called Akismet, and it works similarly to junkmail filters. After a comment (or virtually any piece of content) has been submitted, the Akismet module passes it to a server where it is analyzed. Content labeled as potential spam is then saved for review by the site admin and not posted to the blog.

Pricing

Akismet follows my absolute favorite pricing model. It's free for workaday Joes like me and costs money only if you're a large company that will be pumping lots of bits through the service. They realize that most small bloggers are not making any money on their sites, and they price their service accordingly. Very cool.

Installation

In order to use Akismet, you need to obtain a Wordpress API key. I'm not entirely sure why, but it is free and having a collection of API keys is fun. So get one if you have not already.

The Akismet Drupal module is appropriately named Akismet. It's not currently hosted on Drupal.org, but hopefully the author will eventually host it there as that is where most people find their Drupal modules. Instead, you will need to download the Akismet module from the author's own site. The installation process is standard. Unzip the contents into your site's modules directory, go to your admin/modules page and enable it. There is no need for additional Akismet code as all the spam checking is done on Akismet's servers.

Configuration

After installing Akismet, I was immediately impressed at how professional the module is. There were absolutely no problems after installation. Configuration options are powerful and very well explained. The spam queue is very nice and lets you quickly mark content as "ham" (ie not spam) and delete actual spam. As you build up a level of trust with the spam detection, you can configure the module to automatically delete spam after a period of time.

Spam filtering can be enabled on a per node type basis, allowing you to turn off filtering for node types submitted by trusted users (such as bloggers) and on for others (eg forums users). Comment filtering is configured separately.

Another sweet feature is the ability to customize responses to detected spammers. In addition to being able to delay response time by a configureable number of seconds, you can also configure an alternate HTTP response to the client, such as 503 (service unavailable) or 403 (access denied). Nice touch.

One small problem

I've only been working with Akismet for several days now. And I'd previously been using captcha, which I imagine got me out of the spammers sights for a while (spammers seem to spend most of their efforts on sites where their scripts can post content successfully). So far, Akismet has detected 12 spams, 2 of which were not actually spam. These were very short comments, and I imagine Akismet takes the length of the content into consideration. I assume that as the Akismet server processes more and more pieces of content, it will become more accurate in picking out spam versus legitimate content. Each time a piece of flagged content is marked as "ham", it is sent to Akismet where it can help refine their rule sets and make the service more accurate.

Perhaps Akismet could provide an additional option that allows users to increase or decrease tolerance for spam. I would prefer to err on the side of caution and let comments through.

For a period of time, Akismet was allowing blank replies to go through and I found that both captchas and Akismet worked well together. Not every comment with spam is from a bot, and I found Akismet does a good job blocking the human spammers too. It looks as if Akismet has found the blank comment bug and made corrections. In other words, I may follow your lead and remove my captcha too!

Bryan
CMS Report

The first two comments to this thread were both marked as spam, though I don't really understand why. If Akismet is not able to reduce its false positives, it will prove no more useful than an admin queue. The test continues.

It is odd. I've had 2 or 3 that were unpublished but not marked as spam; but that's it.

You should turn on the extra fields so that your comment posters can enter a name and a URL. I think that might be part of why akismet thinks some of this is spam.

I didn't realize I had those fields turned off. Thanks for the pointer. We'll see if that lessens my Akismet false negatives.

I don't really understand why

Actually the image in the post is really funny :-) I don't like Akismet btw. There's still too much spam. The best thing to do is implement sth individual. No readymade plugin or sth because spammers always find a way to circumvent the most common antispam-stuff ..

Probably the best way to fight against bulk spammers, those that use scripts to find sites and in my case are the annoying ones because they flood my sites with all kinds of useless posts, is to do some javascript tricks on the "post reply" or "register" buttons.

Most scripts can find those buttons automatically and then they display the current webpage to an operator who enter the captcha. The rest of the process is also automatic.

If you make a fake button which only a robot can follow and you change the destination of the link via JS to the real posting page, you can successfully avoid most spammers.

--
Liz
Poemas

I hate captchas. Half the time I can't figure out what they are saying.

This is true! Sometimes the CAPTCHA is too good!

Well, Akismet IS a great thing. They also claim it does not matter which language your blog is in. So we tried on a German Drupal-based multi-blog site and it worked really good. But all of a sudden, nearly all German language content no matter what it really included got marked as spam and frustrated the users of the blog.

So we switched it off since Akismet also did not see any point in replying on our inquiry. We were in fact already considering to use Akismet on a paid enterprise level, so you might thing that they would have had some interest in us, but no.

In fact, we switched back to the older Drupal spam.module and this proofed to be a very good choice anyway. In addition, we have ModSecurity installed on our Apache server which is not an outspoken spam filter, but it watches anything that comes in and goes out even, which does help preventing an automated spamming attack non-the-less, if that system tries to post too aggressively.

Anyway: I hope that Akismet does a better job for you with English language posts then it did for us with German language posts; nontheless, watch the spam list for any false positives, though!

I believe Akisment is a pretty new service. It's understandable if they don't offer multi-language support initially.

But I do think they should definitely respond to inquiries.

please help me i've got spam!!!! hw can it change??? please guys???

I use Akismet on my sites powered by Drupal and WordPress. It's one of the best solutions against spam comments.

I continue to be happy with Akismet. It's very good at detecting most spam. However, one still needs to review comments on a regular basis. I find that a lot of the 'hey great article' (whose purpose is to create a link to another site) spam posts tend to get through.

But so far, I've been able to leave anonymous commenting on.

thanks its for me also,

Yeni sezon

I would have to agree. I us the new WP after upgrading my SQL hard core, which was totally insane, but after that then new WP 2.1 with askimet is awesome.

Just saw that moronic qwerere spammer. I hate this guy and have him blocked on all my sites, but he keeps trying to login every day multiple times. Either its a bot or a login + password was posted to a hacker site. In any event, I was wondering if this site uses askimet because the qwererere guy obviously got through.

My solution was to ban anyone with a .info or .tv or .biz site from creating a user.

just noticed that my akismet service is throwing 503 service unavailable message every now and then, wondering whats wrong.

Akismet works best for me. I used to get 15~30 spam comment everyday and I am tired of that. Thanks to Akismet, I only getting it once every few days now.

I have passed to the new version of wordpress few months ago, and Akismet installed as default, and it really stops spam messages, I was tired because of cleaning viag.ra messages:)

We must all understand that blogs are a great tool for people sharing information and have a voice. There is no software that can stop spam in my opinion

I have this exact same problem I post some stuff on my blog then I go out and make some comments but I think I made a few to many or something becuase all a sudden blogs with askimet are no longer accepting my comments even though every one on my comments is real I read the blogs and the post that I comment on and leave a comment that pertains to the post and in most cases is usefull to the post but like I said maybe I posted 1 to many comments because now it is not letting me post comments on blogs using askimet, does anyone know when this is lifted or why they would call real post spam

Akismet sounds great for dealing with lazy spammers, but with the value of backlinks constantly increasing, there's a lot of motivation for spammers to "kick it up a notch" as Emeril would say.If you can tackle the bots and the lazy spammers you've probably illiminated 95% of the problem.

Great post. You are right on the mark.

Actually, I get some good customers from the spammers. Persistence is the key.

Once I found out how to get customers for my business on a consistent basis, all else was secondary.

Blogging is a big part of that!

I always wonder why some blogs require the reader to create an account before leaving a comment. I must admit that I couldn't be bothered. Maybe I am wrong but isn't the whole point of having a blog to encourage interaction with your readers? Still I guess it is one way of preventing spam.

I like Akismet and use it on multiple Wordpress blogs that I own. It's good to know that it is available for Drupal as I plan on using this CMS in the near future. Just braving myself to take the plunge as I have heard that it can be a bit complex. I like Wordpress because of it's simplicity but I know that Drupal has more functionality so I think it's the way to go.

Very nice module. You say the referral program is not available to Canadians. Would this also include Australia do you think?

To be honest I'm so sick of spam on my blog I have removed the option for users to comment.

In reference to people being to lazy to login to post comment I think "captchas" are fine as long as they are readable, the ones on this site are great, on some other sites they are just ridiculous. Has anyone seen the new ones they have on filefactory (I think) with the dogs and cats? It’s the most ridiculous thing I never seen.

I ended up turning off comments on my dog blog. Once a blog gets really popular, even Akismet does not help enough. I was still getting lots of spam. It is good for more niche blogs that don't get the hordes of idiots, I think.

I would have to agree. I us the new WP after upgrading my SQL hard core, which was totally insane, but after that then new WP 2.1 with askimet is awesome.

http://hardmoneyloans.org