Amusing web crawlers
 

I was merrily blocking a really nasty spambot from keymachine.de tonight and watching the Yahoo bots crawl around the board when I notice some amusing things.

The Yahoo bots attempt to access the download threads. Since they are not logged in, they can't get to them, of course. That's a good thing since otherwise they would be blowing megabytes of bandwidth on each crawl. That's a really strong reason for all sites to require logins and go the extra mile to block bots from creating accounts.

Another bit of amusement was someone from an irresolvable IP address (65.54.154.48) that turned out to be in a Microsoft netblock. I suspect that's a general net portal for employees and a sims fan happened to be surfing the sites at 10:00 PM on a Tuesday night, but still, it's still fun to think that maybe there's a reason why the corporation itself would be interested in the game! :salute:

Yes, that would be fun! (Hey, at least we would know that they had good taste! )

On the subject of Bots/etc (of which I know very little) I can't help but notice that- once again- Sun Sims appears to be free of things that seem to run almost rampant on many other sites.

I am curious as to why that is? It's obvious from being on Sun Sims that it's possible to be Bot/Troll/etc free, yet it seems to be such a huge issue (especially the "Spam Bots") on other sites.

Is it really- in your opinion- massively hard to keep the "Web Trash" off a Web Site, or does it boil down to lack of security blocks and/or lazy Moderators?

-ZW

The price of freedom is eternal vigilance. Thomas Jefferson

Although Thomas Jefferson never heard of spam, he'd probably agree that's the price of freedom from spam too! Basically, early every day, Greg checks for spammers and blows away their accounts and all their spammy posts with one powerful click. If you leave even one for 24 hours, that spammer comes back and brings all his or her little spammer friends. Boards with inactive administrators and moderators (and spam-friendly settings) quickly vanish into piles of spam.

I'm amazed that InSimAdult didn't vanish into spam this spring, but apparently, Kathy got the settings right to keep the spam down, so that the shoestring moderation that was available could handle what did get through.

Sadly, all of Orkut (the social networking group) seems to have vanished for all practical purposes -- it's full of Indians, Pakistanis, and Brazilians who have filled up every unguarded group (unfortunately including the Moon Society group) with spam in languages other than English. I can tell it's Spam, because it's got the Hormel trademark on it! The only reason my Orkut account is at all functional is because I refuse all connections which aren't accompanied by a message in English.

Now if Ben Franklin was still alive, he'd take the Senate out one night, get 'em drunk, and get an anti-spam law passed the next day!

It's a combination of things:

1. As Rose said, I check for spammers every time I log on to the site, at least once a day and often every few hours. If it's there, I deal with immediately and terminate both the spam and the user with extreme prejudice.
   
   
2. I require email address verification. That cuts out a huge number of spambots that aren't sophisticated enough to receive and respond to email.
   
   
3. I can't be bullied into relaxing the security by trolls claiming that Hotmail or Gmail blocks the confirmation email from the board.
   
   
4. I use one of those "enter the secret code from the image" things. That defeats a lot of spambots.
   
   
5. I added an extra question to the signup procedure that cuts out most spambots. You have to agree or disagree with a question.
   
   
6. I block all known spammer domains--hundreds of them, maybe even thousands--so that the web server simply does not respond to queries from those IP addresses.
   
   
7. I block all known domain name and IP address maskers.
   
   
8. I require manual activation of every registration and check every new one to assure that the registrant is connecting to the board from a legitimate ISP. (I do allow Hotmail and Gmail, for now, but the IP address you connect from must be a legitimate, identifiable ISP with responsible spam policies or I won't activate the account.)
   
   
9. I periodically check who is repeatedly hitting the boards without registering. This has added a few more spambot domain IP addresses to the blocked list.
   
   
10. My anti-troll policy is to delete and ban without comment,which only takes a few seconds. That means a troll who is attempting to manually spam the board has to do at least 100 times as much work as I do, and the only reward the troll gets is having spam messages appear for just a few minutes.

I'm not sure lazy is the right word for BBS admins who dont do these things. I suspect that many don't know how and the rest don't realize that it only takes a few minutes of your time to set it up and then a few seconds to deal with anything that gets through.

So instead of lazy, the worst we might say is ignorant or misinformed. Inattentive ought to be on the list, too.

Oh, and one more thing that cuts down on spammers and trolls: We don't advertise this site so I'm sure that most of them don't even know it exists! :laugh:

Yep, and the ones that find us don't bother telling their spammer buddies after their spam vanishes the first time.

Like I said, leave spam up for any amount of time, and more spammers will follow...

The most important thing is to never, never, never click on a spam link. That's how the spammer finds out that he's got a hot one because it reports the IP that the spam is coming from. So if you click on a link to spam from Sun Sims, you've just reported to the spammer that the attack was successful.

(On the other hand, it's been a long time since we've seen a spammer attack on Sun Sims. The lynching committee must be doing its job!)

Footnote: Today I blocked ask.com's crawler at 65.214.45.145 because it was using an access method that looked like an attempt to bypass the site's security system. This generated a series of error messages.

More amusement: At this moment, there are 14 web crawlers musing about on the board and none of them has generated any warnings; so whatever the problem was, it must be unique to ask.com.

Still more amusement: We are blocking access from Limelight Networks in Tempe, Arizona because for several days, bots from their servers have been attempting to read user profiles on this board. I assume it's someone attempting to harvest email addresses for spam.

As best I can tell, Limelight Networks is not a legitimate ISP and none of our members have ever signed on from there, so this should not inconvenience any real Sims fans. If I'm wrong about that, please let me know.