For many, many years, I had been using Spam Karma 2 for comment, pingback, and trackback spam protection here on my blog and it seemed to do a decent job. The idea behind SK2 is to check for things like Javascript (few bots have JS enabled), the number of links in the post (spam obviously has lots usually), how long it took them to post the comment (bots post it nearly instantly), and so forth. Each property would add or subtract karma points from the comment until a final score was given. The comment would then be marked as ham (good) or spam (bad) based on that score.
It of course wasn’t perfect. Spam bots are getting smarter and a plugin such as SK2 alone just couldn’t be accurate enough. So, a year or so ago, I added an Akismet plugin for SK2. The idea is the plugin talks to Akismet and gets a ham or spam rating for the comment. This result is then given to SK2 to use in it’s overall score. It worked great, at least for a while.
However, even with the help of Akismet, SK2 just wasn’t cutting it lately. Akismet was too often saying it was ham when it really was spam or on occasion, the other way around (that ham was spam).
So, two weeks ago today, I decided to give Defensio a try.
It’s very similar to Akismet in how it works from the standpoint of the user. Every comment, pingback, and trackback is automatically submitted to Defensio’s servers for evaluation. However, the big difference, besides the increased accuracy I’ve observed (more on that in a minute), is that each comment is given a “spaminess” rating. For comments that Defesnsio feels is for sure spam, that spaminess rating will be 100% and less for ones that it’s really sure on. This makes it incredibly easy to make sure Defensio is doing a good job. You just visit Defensio’s comment spam page on your blog and glance at the short list of non-obvious spam. It hides all the obvious spam, although you can show them if you really want to.
Another great thing about Defensio is that it learns on a per-blog basis, something that I don’t believe Akismet does. Yes, Akismet uses data from everyone else out there to help learn what spam is and isn’t, but Defensio also learns about your blog specifically. For example, if you blog about Viagra (what an odd blogging subject), you’d obviously get a ton of comments about Viagra. Well, Akismet may for example think these comments are spam since Viagra is a common spam subject. Defensio would learn as you correct it and eventually get better. As they say on their site, “one person’s ham might be another person’s spam”.
They also have a section on their website where you can go to view graphs of your total spam, total ham, and accuracy. However, I had some weird bugs with their graphs. I had two March 23rd’s with the 2nd showing zero (hence the dip) as well as a spike for some reason on about the 3rd. (The large increase on the 2nd was real — my server was hammered with spam on that day.)
So, now to the important part — how did Defensio do on my blog over the past two weeks? Well, quite good actually even though they say it may take a week or two to fully adapt to your site. Here’s the stats from my Defensio spam page for the past two weeks:
- Recent accuracy: 99.86%
- 10628 spam
- 116 legitimate comments
- 9 false negatives (undetected spam)
- 6 false positives (legitimate comments identified as spam)
Okay, so it did indeed get 6 false positives, but they were very easy to find as they were right there at the top of Defensio’s page since the spam is sorted by spaminess.
So, will I continue to be using Defensio on blog? Definitely, and I recommend you do too.
Wow that’s an awesome accuracy and a -huge- amount of spam that you’re getting there. Good to see that we’re doing good on your blog!
Carl Mercier on April 6th, 2008 at 7:03 AM wrote:
Yeah, 9 outta 10.6k is friggin’ awesome. I’m quite happy I made the switch! 🙂
And no clue as to why I’m such a popular target for spammers. I guess it’s my high page rank (6 last I looked) combined with my blog being relatively old.
A lot of spam means that you’re doing something right 🙂
You should definitely show the world how much spam you’re getting with the Defensio badge. Instructions are in the readme file.
Nice review. I’m getting similar amounts of junks and so far I’ve been rather happy with Akismet, yet indeed some quite obvious spams sometimes manage their way in. Might give defensio a try some day.
(To be honest their plugin is mostly coded like crap)
Carl Mercier on April 6th, 2008 at 7:19 AM wrote:
Yeah, I was just waiting to see if I was actually going to stick with Defensio or not. 🙂
Ozh on April 6th, 2008 at 10:18 AM wrote:
Yeah, Akismet’s accuracy is far from bad, but as you said, it seemed to miss quite a few obvious spams, something I just couldn’t have.
Ozh on April 6th, 2008 at 12:40 PM wrote:
Heh, I noticed the same thing, but it works fine with no errors, so I can easily overlook the code behind the scenes. 🙂
We’ve been working at improving the plugin lately, but as Viper says, it works great as it is right now!
I don’t know how Defensio works for you, my defensio accuracy is constantly dropping from 98 to 96%.
And its dropping ever since. I am back to Akismet.
NSpeaks on April 7th, 2008 at 8:03 AM wrote:
At the bottom of my initial post, I posted my stats. 😉
I meant….howcome its working for you when I have never seen its accuracy touch 99%. For me…accuracy was 96% when I last used it 2-3 days ago. I am now on Akismet.
I switched back to Akismet last week after having tried Defensio since shortly after it’s release. I have a small niche blog in Dutch, that’s only getting spammers attention since a few weeks, but Defensio’s accuracy was still arround 95% after all those months (best was around 96%), and hadn’t learned yet that pingback’s weren’t necessarily spam.
And it worked so slow… Akismet’s page load consideralably faster. For the near future, I’ll stick to Akismet.
Thanks, good to know there is an alternative out there. So far akismet is cutting it for me, but recently I have noticed a number of comments slipping through even with Bad Behavior installed. Spam…what fool invented that eh? 😛
I was using Akismet, Spam Karma, and Bad Behavior on our family’s sites. WAY too much spam was getting through, despite those measures, so I tried Defensio as soon as I heard about it, back in January.
My statistics start January 21, and while there was a bit of a learning curve, the accuracy has never been below 98%. After the first few weeks, it has stayed at or better than 99% except for a brief dip during a barrage of spam a few weeks back.
I don’t get nearly as much spam as you do, despite having had my domain since 1995 or so. Defensio says there’ve been 7601 spam comments/trackbacks since January. I’m glad not to be in your league!
We had a wiki on one of our sites, but it got eaten up with spam, despite everything my partner did trying to secure it. We have another (can’t remember the software) up now that hasn’t been attacked, but the URL isn’t linked from anywhere, so it isn’t really a fair comparison.
Interesting.
I haven’t tried Defensio (yet)
I started my blog about 3 years ago, and once I switched to wordpress, I quickly discovered akismet (its built-in to wordpress).
Although it worked well, I noticed it would add *significantly* to my websites bandwidth usage (all that communication with the akismet server).
So I tried bad behaviour about 1 year ago.
I found it worked brilliantly (I even made a paypal donation to the author).
BB is currently blocking about 500 spam comments per day. The ones that slip past BB get picked up by akismet (about 100 per day).
I still get the occasional spam (maybe once a month) by people manually spamming.
If the spammers get smart, and find a way through both BB and akismet, then I’ll take a good look at defensio. I like the “learning from your blog” part, and the spamminess rating.
I used to use Bad Behavior, but I didn’t want to deal with any false positives as it would literally block people from my site. I can deal with recovering a false positive comment or marking a comment that slipped through Defensio, but I can’t do anything about BB being wrong.
Pingback: Decided to give Defensio Anti-Spam a shot… over Akismet » Chaos Laboratory