Page 1 of 1

How to block an abusive bot? (majestic12.co)

Posted: 20 Jan 2016, 11:41
by alexio
We're getting hit hard by the infamous (and probably nefarious) "majestic12.co.uk/bot".

I've added the various IP's it uses to "Deny", but am wondering...:

Is there a way to block ALL requests coming from "majestic12.co.uk/bot"?

Thanks in advance...

Re: How to block an abusive bot? (majestic12.co)

Posted: 20 Jan 2016, 12:40
by marcele
Place the following file:
Centos / RHEL
/etc/httpd/conf.d/bad_bot.conf

Debian / Ubuntu
/etc/apache2/conf.d/bad_bot.conf

Content

Code: Select all

<IfModule mod_setenvif.c>
# SetEnvIfNoCase User-Agent ^$ bad_bot
SetEnvIfNoCase User-Agent "^MJ12bot" bad_bot
SetEnvIfNoCase User-Agent "^MJ12bot/v1.4.5" bad_bot

# Apache < 2.4
<IfModule !mod_authz_core.c>
  <FilesMatch "(.*)">
    Order allow,deny
    Allow from all
    Deny from env=bad_bot
  </FilesMatch>
</IfModule>

# Apache >= 2.4
<IfModule mod_authz_core.c>
  <FilesMatch "(.*)">
    <RequireAll>
      Require all granted
      Require not env bad_bot
    </RequireAll>
  </FilesMatch>
</IfModule>
</IfModule>
Restart Apache! This will block MJ12bot useragent.

Re: How to block an abusive bot? (majestic12.co)

Posted: 06 Dec 2017, 12:23
by wonderwall
Will this also work in an Nginx+Apache setup?

Re: How to block an abusive bot? (majestic12.co)

Posted: 06 Dec 2017, 14:42
by marcele
wonderwall wrote: 06 Dec 2017, 12:23 Will this also work in an Nginx+Apache setup?
Yes it would.

Re: How to block an abusive bot? (majestic12.co)

Posted: 06 Dec 2017, 16:32
by wonderwall
Great - thanks for the quick reply.