regex for badbots not triggering after updating to v9.10

This forum is only for reproducible bugs with csf and lfd (i.e. not iptables problems, lack of understanding how to use a feature, etc). Posts must be accompanied with full technical details of the problem and how it can be recreated. Any posts not adhering to this, or not considered bugs, will be moved to the General Discussion (csf) forum.
Post Reply
csmaniac
Junior Member
Posts: 2
Joined: 12 Jul 2016, 08:35

regex for badbots not triggering after updating to v9.10

Post by csmaniac »

Dear Support,
after updating to csf v9.10 custom regex rules stop triggering regex.custom.pm
i use centos 7 with plesk and mod_Security

regex.custom.pm rule:

Code: Select all

      if (($lgfile eq $config{CUSTOM1_LOG}) and ($line =~ /(\S+) - - \[\S+\s+\S+\] "(GET|POST) \/ HTTP\/\S+" \d+ \d+ \S+\s+"\S+ \(compatible\; (MJ12bot|Baiduspider|AhrefsBot)\/\S+ \S?http\S+\)"/)) {
         return ("GET POST HEAD Attempt From: $1","MJ12bot-Baiduspider-AhrefsBot","1","1");
    }
CUSTOM1_LOG: /var/www/vhosts/system/*/logs/access_*log

Lines to trigger:

Code: Select all

149.202.48.192 - - [12/Jul/2016:08:43:51 +0300] "GET / HTTP/1.0" 200 96340 "-" "Mozilla/5.0 (compatible; MJ12bot/v1.4.5; http://www.majestic12.co.uk/bot.php?+)"
180.76.15.30 - - [12/Jul/2016:08:24:38 +0300] "GET / HTTP/1.0" 200 21132 "-" "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)"

and

Code: Select all

     if (($lgfile eq $config{CUSTOM1_LOG}) and ($line =~ /(\S+) - - \[\S+\s+\S+\] "(GET|POST) \/\S+ HTTP\/\S+" \d+ \d+ \S+\s+"\S+ \(compatible\; (MJ12bot|Baiduspider|AhrefsBot)\/\S+ \S?http\S+\)"/)) {
         return ("GET POST HEAD Attempt From: $1","MJ12bot-Baiduspider-AhrefsBot2","1","1");
    }
CUSTOM1_LOG: /var/www/vhosts/system/*/logs/access_*log

Lines to trigger:

Code: Select all

180.76.15.134 - - [12/Jul/2016:08:35:17 +0300] "GET /language/en/kallirroe-hotel/ HTTP/1.0" 200 7285 "-" "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)"
51.255.65.8 - - [12/Jul/2016:08:54:14 +0300] "GET /tag/%CF%80%CE%BB%CE%B7%CF%81%CF%8E%CE%BC%CE%B1%CF%84%CE%B1/ HTTP/1.0" 200 96452 "-" "Mozilla/5.0 (compatible; AhrefsBot/5.1; +http://ahrefs.com/robot/)"
173.208.157.186 - - [12/Jul/2016:08:38:10 +0300] "GET /shop/ HTTP/1.0" 200 18182 "-" "Mozilla/5.0 (compatible; MJ12bot/v1.4.5; http://www.majestic12.co.uk/bot.php?+)"
Best Regards
ForumAdmin
Moderator
Posts: 1523
Joined: 01 Oct 2008, 09:24

Re: regex for badbots not triggering after updating to v9.10

Post by ForumAdmin »

Instead of using

Code: Select all

($lgfile eq $config{CUSTOM1_LOG})
try using this instead:

Code: Select all

($globlogs{CUSTOM1_LOG}{$lgfile})
csmaniac
Junior Member
Posts: 2
Joined: 12 Jul 2016, 08:35

Re: regex for badbots not triggering after updating to v9.10

Post by csmaniac »

Hi, I have already tried it and unfortunately it didn't work.
I have also re-installed csf v9.10 from scratch and rebooted server.
When i enter multiple directories path in custom_logs it doesn't watch those logs. in fact it watches some of those logs randomly! Sometime it blocks and sometimes it doesn't block. I can see it reading and searching those logs.
CUSTOM1_LOG: /var/www/vhosts/system/*/logs/access_*log

On another server I don't use multiple directories path in custom_logs I use single directory path and it works fine!
I think something happens when i enter a path with multiple directories to watch.
This specific path has about 80 directories inside vhosts/*/logs for csf to watch!

Any ideas?
Regards
Post Reply