edin.no-ip.com

Since a long days before I keep on using Apache's mod_access for spam or bad robot filtering (http://edin.no-ip.com/content/block-apache-visiting-abnormal-user-agent). It is quite handy and simple; BTW, you need to configure it manually. The benefit of the model is you only need to have a functional Apache installed then you can set it up without any special difficult and dependence; and the drawback is simple that it is not flexible.

As Debian's Fail2ban already come with apache-badbots.conf, why not utilize it? As it will function in firewall level, rather than application level (Apache), using this model would be more secure and stable, too.

Setting this up is very simple. In case of Debian, install Fail2ban with:

Then check /etc/fail2ban/filter.d/apache-badbots.conf and you will find a well pre-defined blocking list, which fetched from http://www.user-agents.org. What you need to do is active this filtering rule within your Fail2ban configuration. As mentioned in the header section of /etc/fail2ban/jail.conf, we should create a file called as /etc/fail2ban/jail.local which contain our changes for override, e.g.:

After restart Fail2ban with /etc/init.d/fail2ban restart, check your iptables with iptables -nvL and you should have similar result:

In order to check for ban/unban record, try cat /var/log/fail2ban.log | grep WARNING. E.g. soon after I have install Fail2ban it catch 1 IP for ban, due to rule of apache-overflows: