How to Block Bots from Seeing your Website – Bad Bots and Drive-by Hacks Explained

You’ve been hacked?
Don’t take it personally. Most websites are hacked randomly and with total disregard to your person, position, or title. It’s just business…

Kevin Mitnick once said, “The hacker mindset doesn’t actually see what happens on the other side, to the victim.” Born true a decade ago and still does today.

Kill the Bot

Kill the Bot!

So why are you allowing your website to be “owned?”

Prevention is your best weapon, but most people seem oblivious to how easy it is to blend into the virtual masses and hide in the shadows online.

One method of blocking hackers involves blocking bots from even seeing your website. If a bot stumbles onto your website and is immediately repelled, you’ve just scored a big win security-wise. Don’t be a target, be a bot killer!

 

How to block bots from seeing your website

Blocking bots is crazy-simple to do. First you’ll need to know how to locate your .htaccess file is a file most often found in your public_html directory. It’s just a text file. Don’t be scared… :).

Most web hosts have a basic File Manager, where the list of your website’s files can be viewed, along with an “Edit” button to make and save changes.

With your trusty File Manager or FTP File Editor in hand, you are less than one minute from being done with this tedious task.

Below is the the bot blocking list I use. Feel free to edit it to your heart’s content:

 
Within this list you’ll see sections with headings like “Yandex.” If you are a Yandex fan then remove the required lines of text and you’ll be all set.

Likewise, if your site is being hammered by fake Bingbots, just replace any of the existing lesser known bots in the list with “bingbot.” For example, replace:*
RewriteCond %{HTTP_USER_AGENT} ^SuperBot [NC,OR]
with
RewriteCond %{HTTP_USER_AGENT} ^Bingbot [NC,OR]
*Just remember to remove this line in a week or so (once you feel the attack has run it’s course).

 
In summary:

  1. First, see the list using the above link.
  2. Copy it to your computer’s memory.
  3. Next open your .htaccess file via your editor.
  4. Paste this list at the very top of your .htaccess file.

Save.
Done!
Seriously, you are done.

 
Check your website to see if it’s loading properly and without error. If so, then you’ve just built the poor man’s bot firewall in all of one minute flat, and can sleep better at night knowing you are super smart, accomplished, and can say with pride, “I am so smart!

 

 

 

Summary
Article Name
How to Block Bad Bots from Seeing your Website - Bad Bots and Drive-by Hacks Explained
Author
Description
Jim Walker, The Hack Repair Guy and website security expert explains bad bots and drive by hacks in this easy to understand how to article. Jim is the author of the block bad bots blacklist used by thousands to secure their websites.

About Jim Walker

Jim Walker has been a website security expert and website hosting services provider for over 16 years. Living in San Diego, California, his current passion is everything WordPress. He manages HackRepair.com, a malware cleanup and website security services company, and HackGuard.com, a WordPress security and WordPress management service.

Your Comments or Questions for The Hack Repair Guy?

Loading Facebook Comments ...
Loading Disqus Comments ...

Comments

  1. alphadarkriz says

    I’m having an issue with one bot in particular which is not a malicious bot, but rather a bot used by an SEO tool. The name is MJ12bot from MajesticSEO. I can see it here:

    RewriteCond %{HTTP_USER_AGENT} MJ12bot [NC,OR]

    I don’t even have the blacklist feature from hackrepair enabled on one site and it is still being blocked. Is there a work around for this? Can I edit this one line out somewhere?

    Thanks

  2. Bill Smithem says

    One of the drawbacks of using the list is that some hosting companies are now recommending disabling the wp-cron mechanism and using real cron jobs to trigger wp-cron instead. The recommended method I’ve seen is to wget cron.php and wget is one of the user agents disabled when you use the hackrepair list in iThemes Security. Cron never runs, and with the options recommended, you don’t see any errors.

    I’ve stopped using the list because I don’t want to discover that my sites haven’t been backed up for months because the user agent I’m using to run the jobs has been added to the list.

    • Jim Walker says

      wget is not part of the latest list (though it was last year).
      If it is, then it’s a simple matter of removing that line from the .htaccess file to resolve.

      If I’m misunderstanding your question please reply back and I’ll do my best to better answer your question.

  3. Robert says

    Below is copied from my .htaccess file. Since the “RewriteEngine On” is already there, I commented out that
    line from the list I copied from your site. Will this still work effectively? Is there any way for me to tell if the Bad Bots are being blocked?

    Thanks

    # BEGIN iThemes Security
    # BEGIN Tweaks

    RewriteEngine On

    # Begin HackRepair.com Blacklist
    # RewriteEngine on
    # Abuse Agent Blocking

    • Jim Walker says

      The only way you can tell if bad blocks are being blocked is via your server or account log file.

  4. marga says

    Hello, thanks for the list. I´m using I themes security, in wordpress. But I have some issues.
    Since a couple of months I have really a lot of bad logins attemps., I have already ca 400 bad ip adresses.
    1 Is this normal?
    since yesterday there a 30 new ones. I have to put them all manually on the banned hosts list.
    2.Do you know if there is a way to do this automatically?
    3. Are you interested in the list?
    I never got an answer on this in the wordpress forum.
    thanks
    greetings Marga

    • Jim Walker says

      Receiving “many” bad login attempts?
      That’s awesome!

      It just means the plugin is working. Personally I would ignore these and give up blocking IP’s.
      The plugin is doing its job in blocking the bad bots for you, so worrying over what IP attempted to login is totally unnecessary- well, unless you enjoy worrying (…I’m not judging – to each his/her own). :)

  5. Elton says

    How do you know you’re site is being spammed by bots? like for example I have a wordpress site and the iThemes Security (formerly Better WP security) plugin.

    • hackrepair says

      Well, that is a good question. Generally you can tell the difference between a person or a machine posting trash comments. As for whether people can write badly and post totally not relevant stuff as well–I suppose that’s true too. :)

  6. Brijesh Tejani says

    Hi, the WordPress Plugin iThemes Security (formerly Better WP security) is offering your bad bot list saying: “As a getting-started point you can include the excellent blacklist developed by Jim Walker of HackRepair.com.”

    What I was wondering is that are any drawbacks associated with blocking (bad) bots. I’d appreciate a detailed response/explanation.

    Thanks.

    • hackrepair says

      I have received virtually no complaints over the past two years. The bad bot list is really quite minimal, blocking only the most well known and most egregiously aggressive bots.

Leave a Reply

You must fill in your Livefyre SiteID in the Comments Evolved plugin options.