Brute Force Detection

Current Release:
http://www.rfxn.com/downloads/bfd-current.tar.gz
http://www.rfxn.com/appdocs/README.bfd
http://www.rfxn.com/appdocs/CHANGELOG.bfd

Description
BFD is a modular shell script for parsing application logs and checking for authentication failures. It does this using a rules system where application specific options are stored including regular expressions for each unique auth format. The regular expressions are parsed against logs using the ‘sed’ tool (stream editor) which allows for excellent performance in all environments. In addition to the benefits of parsing logs in a single stream with sed, BFD also uses a log tracking system so logs are only parsed from the point which they were last read. This greatly assists in extending the performance of BFD even further as we are not constantly reading the same log data. The log tracking system is compatible with syslog/logrotate style log rotations which allows it to detect when rotations have happened and grab log tails from both the new log file and the rotated log file.

You can leverage BFD to block attackers using any number of tools such as APF, Shorewall, raw iptables, ip route or execute any custom command. There is also a fully customizable e-mail alerting system with an e-mail template that is well suited for every day use or you can open it up and modify it. The attacker tracking in BFD is handled using simple flat text files that are size-controlled to prevent space constraints over time, ideal for diskless devices. There is also an attack pool where trending data is stored on all hosts that have been blocked including which rule the block was triggered by.

In the execution process, there is simply a cron job that executes BFD once every 3 minutes by default. The cronjob can be run more frequently for those that desire it and doing so will not cause any performance issues (no less than once a minute). Although cron execution does not permit BFD to act in real time, the log tracking system ensures it never misses a beat in authentication failures. Further, using cron provides a reliable frame work for consistent execution of BFD in a very simplified fashion across all *nix platforms.

Funding:
Funding for the continued development and research into this and other projects, is solely dependent on public contributions and donations. If this is your first time using this software we ask that you evaluate it and consider a small donation; for those who frequent and are continued users of this and other projects we also ask that you make an occasional small donation to help ensure the future of our public projects.

Trackbacks (47)
Comments (48)

#1 written by Ovidiu
about 1 year ago

Reply Quote

also I’d like to know if the number of allowed failures is related to a certain time frame, i.e. 10 failures / 10 minutes?

# how many failure events must an address have before being blocked?
# you can override this on a per rule basis in /usr/local/bfd/rules/
TRIG=”10″
- #2 written by Ryan M.
  about 1 year ago
  
  Reply Quote
  
  The max failures (TRIG) is per IP address and is an incrementing value across each BFD execution, so even if you have BFD run every 1 minute or 10 minutes, it will track the authentication failures for an IP address in total and when it reaches TRIG failures, it gets banned.
#3 written by Ovidiu
about 1 year ago

Reply Quote

I am trying to use BFD instead of fail2ban but I am unsure about it because I can’t seem to find a setting as to how long a misbehaving hostname is being blocked.
And does BFD act similar to fail2ban and unblocks a certain hostname after a specific amount of time?
- #4 written by Ryan M.
  about 1 year ago
  
  Reply Quote
  
  BFD does not have any features at present to remove a ban after X period of time.
#5 written by Ioan Indreias
about 1 year ago

Reply Quote

Hello Ryan,

Please consider to remove the filter command for the content of the BAN_COMMAND (line 180 in the bfd version 1.5): BAN_COMMAIND=`echo $BAN_COMMAND | tr -d ‘\\&;|’`

We are using the following BAN_COMMAND and we have to delete the filter from the bfd script:

BAN_COMMAND=”(/sbin/iptables -n -L | grep DROP | grep $ATTACK_HOST) || /sbin/iptables -I INPUT -s $ATTACK_HOST -j DROP”

Off course we could put there an external script but we are proffering to not add another layer of bash commands.

BR,
Ioan
#6 written by Jez
about 1 year ago

Reply Quote

Hi,

I’m very happy with your AFP and BFD tools, but I have a question about configuring BFD: by default, conf.bfd has:

BAN_COMMAND=”/etc/apf/apf -d $ATTACK_HOST {bfd.$MOD}”

This adds the hostname of the the attacker to the ban list, but is there a way to add their IP address instead?
#7 written by zwerfkat
about 1 year ago

Reply Quote

Found a small bug in BFD 1.4. Somewhere at line 153 is listed:

if [ "$ATTACK_COUNT" -gt "$TRIG" ] || [ "$ATTACK_COUNT" -eq "$TRIG" ] && [ "$HOST_IGNORE" == "0" ]; then

However, when $HOST_IGNORE is null, $ATTACK_COUNT is not defined at all. This will result into a script error:

[: : integer expression expected

Defining ATTACK_COUNT somewhere at the start of the script will fix this:
ATTACK_COUNT=0

Not sure how the working of BFD is affected by this error…
#8 written by zwerfkat
about 1 year ago

Reply Quote

Roland:

great script, however i’m getting 1000′s of these in my exim logs and it doesn’t ban them:
2011-06-28 05:47:38 login authenticator failed for (ylmf-pc) [113.65.143.13]: 535 Incorrect authentication data (set_id=web)
does anyone have a rule for exim that will ban these as well?
thanks in advance
roland

I am using the following rule for this:

# failed logins from a single address before ban
# uncomment to override conf.bfd trig value
TRIG=”50″

# file must exist for rule to be active
REQ=”/usr/sbin/exim”

if [ -f "$REQ" ]; then
LP=”/var/log/exim/mainlog”
TLOG_TF=”exim1″

## EXIM attacks
ARG_VAL=`$TLOG_PATH $LP $TLOG_TF | grep -E “login authenticator failed for” | grep -E ‘[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+’ | tr -d ‘[]‘ | sed -n -e ‘s/.*login authenticator failed for $[^ ]*$ .* $[\.0-9]*$: 535 Incorrect authentication data.*/\2:\1/p’`
fi
#9 written by Carl
about 1 year ago

Reply Quote

I am very pleased with all the articles or blog posts on your web blog. I get numerous thoughts to help me.
#10 written by Biruny
about 1 year ago

Reply Quote

Hello,

Thank you for this great project
I use it but I find that the system detect the attack but it doesn’t really apply the ban command when it run by the cron job and when I run bfd -q or bfd -s it works and apply the command. by the way I change the ban command to blackhole the IP “”BAN_COMMAND=”ip route add blackhole $ATTACK_HOST”
Also there is a problem in the system that if the IP in the ban.list file the system ignore any attack from the IP, this happen when the system detect the ip but the command didn’t applied

Also I see that the IPs in pan.list file removed after some time, how does this configured and what’s the default time.

Thanks
- #11 written by Biruny
  about 1 year ago
  
  Reply Quote
  
  Biruny:
  
  Hello,
  Thank you for this great project
  I use it but I find that the system detect the attack but it doesn’t really apply the ban command when it run by the cron job and when I run bfd -q or bfd -s it works and apply the command. by the way I change the ban command to blackhole the IP “”BAN_COMMAND=”ip route add blackhole $ATTACK_HOST”
  Also there is a problem in the system that if the IP in the ban.list file the system ignore any attack from the IP, this happen when the system detect the ip but the command didn’t applied
  Also I see that the IPs in pan.list file removed after some time, how does this configured and what’s the default time.
  Thanks
  
  Hello,
  
  any update about these comments ??
  dose any one know the problems ?
- #12 written by Ryan M.
  about 1 year ago
  
  Reply Quote
  
  Use the absolute path to the command in the BAN_COMMAND value, such as /sbin/ip instead of just ip.
#13 written by Salvatore L
about 1 year ago

Reply Quote

I have made a few scripts for some very-specific services. You shouldn’t have to be using Debian, but they are confirmed working in Debian with all the latest packages. For SASL-based authentication failures in Postfix, try the following:

– BEGIN /usr/local/bfd/rules/postfix –

# failed logins from a single address before ban
# uncomment to override conf.bfd trig value
#TRIG=”50″

# file must exist for rule to be active
REQ=”/usr/sbin/postfix”

if [ -f "$REQ" ]; then
LP=”/var/log/mail.log”
TLOG_TF=”postfix”

## Postfix dictionary attacks
ARG_VAL=`$TLOG_PATH $LP $TLOG_TF | grep -iwE “SASL LOGIN authentication failed:|SASL PLAIN authentication failed:” | grep -E ‘[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+’ | cut -d ‘ ‘ -f 7 | tr -d ‘[a-z][A-Z]\[\]\:’`
fi

– END –

Also, if you have your postfix daemon properly configured to block invalid HELO tries, the user actually won’t even make it as far as authentication when faking HELOs, but you will get lots of annoying log entries about it. Here’s a script called postfix2 to also block many failed HELO attempts:

– BEGIN /usr/local/bfd/rules/postfix2 –

# failed logins from a single address before ban
# uncomment to override conf.bfd trig value
#TRIG=”50″

# file must exist for rule to be active
REQ=”/usr/sbin/postfix”

if [ -f "$REQ" ]; then
LP=”/var/log/mail.log”
TLOG_TF=”postfix2″

## Postfix failed HELOs
ARG_VAL=`$TLOG_PATH $LP $TLOG_TF | grep -iwE “lost connection after RCPT from” | grep -E ‘[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+’ | awk -F \[ '{ print $3 }'| tr -d '[]‘`
fi

– END –

Finally, for vsftpd failures, try this script that I borrowed from someone else and modified slightly:

– BEGIN /usr/local/bfd/rules/vsftpd –

REQ=”/usr/sbin/vsftpd”
if [ -f "$REQ" ]; then
LP=”/var/log/vsftpd.log”
TLOG_TF=”vsftpd”
TRIG=”6″

## VSFTPD
ARG_VAL=`$TLOG_PATH $LP $TLOG_TF | grep -w ‘FAIL LOGIN:’ | tr ‘[]‘ ‘ ‘ | tr -d ‘()’ | awk ‘{print$12″ “$8}’ | tr -d ‘:’ | tr -d ‘”‘ | awk ‘{print$1″:”$2}’ | grep -E ‘[0-9]+’`
fi

– END –

Hope those help someone.
- #14 written by Salvatore LaMendola
  about 1 year ago
  
  Reply Quote
  
  Found an error in ‘postfix’ that would cause invalid addresses when the hostname had numbers (kept the numbers and dots so it usually looked something like 207..123.45.67.89. Here is the new and improved postfix script, which works with numbered hostnames:
  
  – BEGIN /usr/local/bfd/rules/postfix –
  
  # failed logins from a single address before ban
  # uncomment to override conf.bfd trig value
  #TRIG=”50″
  
  # file must exist for rule to be active
  REQ=”/usr/sbin/postfix”
  
  if [ -f "$REQ" ]; then
  LP=”/var/log/mail.log”
  TLOG_TF=”postfix”
  
  ## Postfix dictionary attacks
  ARG_VAL=`$TLOG_PATH $LP $TLOG_TF | grep -iwE “SASL LOGIN authentication failed:|SASL PLAIN authentication failed:” | grep -E ‘[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+’ | awk -F ‘\[' '{ print $3 }' | tr -d '[a-z][A-Z]\[\]\:’`
  fi
  
  – END –
  
  Tested on Debian (sid)’s postfix, and a few others.
  - #15 written by Ryan M.
    about 1 year ago
    
    Reply Quote
    
    Thank you for your contribution, added to 1.5 release that is now live, credit included in CHANGELOG. I removed the escape on the -F ‘\[‘ as it was being treated as just ‘[‘ anyways and awk would throw a warning about it under RHEL awk versions.
#16 written by Roland
about 1 year ago

Reply Quote

hi remco

this works for me for dovecot you can give it a try:

REQ=”/usr/sbin/dovecot”
if [ -f "$REQ" ]; then
LP=”/var/log/maillog”
TLOG_TF=”pop3″
TRIG=”8″

## pop3
ARG_VAL=`$TLOG_PATH $LP $TLOG_TF |grep dovecot |grep -w “failed” |grep auth |grep -o -E ‘[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+’`
fi
#17 written by Remco
about 1 year ago

Reply Quote

Hi,

Is there some one that can make a working rule for dovecot?

Thanks a lot.
#18 written by Roland
about 1 year ago

Reply Quote

great script, however i’m getting 1000′s of these in my exim logs and it doesn’t ban them:
2011-06-28 05:47:38 login authenticator failed for (ylmf-pc) [113.65.143.13]: 535 Incorrect authentication data (set_id=web)

does anyone have a rule for exim that will ban these as well?

thanks in advance

roland
#19 written by Justin
about 2 years ago

Reply Quote

Hey are there any rules specific to Plesk?
#20 written by Ray Leventhal
about 2 years ago

Reply Quote

Hi Ryan,

I’m wondering if it would be possible to use bfd with the uw-imap daemon. My server uses this as part of the Parallels Pro (formerly Ensim) control panel and we’re getting bombarded with imap floods.

Thanks for a great set of products and for your ongoing work.

-Ray
#21 written by Hosting Reseller
about 2 years ago

Reply Quote

RFXN Team is the saving grace!
#22 written by Paul Krikom
about 2 years ago

Reply Quote

I am a major fan of rfxn.

BFD actually saved my small company. I use APF & BFD and without it, I would be out of business because I can not afford the expertise that rfxn possesses.

Thank you to the rfxn team! My donation en-route.
#23 written by Gilles
about 2 years ago

Reply Quote

Thanks Ryan. I’ll give Ash a try, then.

Fail2Ban won’t do on this appliance because it’s written in Python and there isn’t enough RAM to fit.
#24 written by Gilles
about 2 years ago

Reply Quote

Hello

I’m looking for a good way to ban IP’s trying to register with my Asterisk server illegally, and was thinking of installing BFD to work with iptables.

Problem is, it’s an appliance running uClinux and Busybox, which provides the following shells: ash, hush, lash, and msh.

Will BFD work with those shells?

Thank you.
- #25 written by Ryan M.
  about 2 years ago
  
  Reply Quote
  
  Ash is a bash compatible shell, so it should work, you may just need to symlink /bin/ash to /bin/bash (ln -s /bin/ash /bin/bash). You may also want to check out fail2ban though not sure how well it will install in a busybox setup: http://www.voip-info.org/wiki/view/Fail2Ban+%28with+iptables%29+And+Asterisk
  
  If you require help creating the asterisk rules for BFD, let me know and I’ll gladly take a stab at it.
  - #26 written by indreias
    about 2 years ago
    
    Reply Quote
    
    Asterisk BFD rules could be obtained from our site – SIP and IAX rules have been tested for Asterisk 1.2 and 1.4.
    
    http://www.modulo.ro/Modulo/downloads/tools/tenora.bfd.tar.gz
  - #27 written by Tom L.
    about 11 months ago
    
    Reply Quote
    
    I know this is an old entry but did anyone ever get rules put together for this? I would be very interested in see them or helping make them. I am currently running the FreePBX distro with Asterisk Ver. 1.8.12.0.
    - #28 written by Ryan M.
      about 11 months ago
      
      Reply Quote
      
      Public release of BFD has asterisk rules included now, so you should be good to go.
#29 written by Indreias
about 2 years ago

Reply Quote

Hello,

Running with bfd 1.4 and found a “minor bug” in bfd script:
ATTACK_COUNT=$[RET_COUNT+ATTACK_COUNT] should be
ATTACK_COUNT=RET_COUNT (line #150)
otherwise the attack is considered valid (ATTACK_COUNT -ge TRIG) as the ATTACK_COUNT lines have been already written into the track.attack file (line #136)

Best regards,
Ioan
#30 written by Ashlynne
about 2 years ago

Reply Quote

Hi, I am trying to write a rule for dovecot.
I created this:
ARG_VAL=`$TLOG_PATH $LP $TLOG_TF |grep ‘mail dovecot: pop3-login: Disconnected:’ |awk ‘{print$10}’ | grep -o \
-E ‘[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+’`

When I run it against the maillog, the hackers IP is listed over and over, but when I create the above rule and I run /usr/local/sbin/bfd -s
the rule runs but it does not capture the hacker’s IP. Could you please explain what the rule should return or how to write a new rule or just point me to a doc where you have described what the rule should be returning to the bfd script?
Thank you
- #31 written by Ashlynne
  about 2 years ago
  
  Reply Quote
  
  Ok, I was able to figure out the rules must return the IP:user to the bfd script.
  Example:
  50.7.246.130:admin
  
  Knowing this I can at last play with creating new rules. I found two problems with the bfd script while creating my new rule.
  1. the TRIG variable is reset to the default value from the conf.bfd file after the first time the rule’s TRIG value is met because the conf.bfd is reloaded. To fix this I moved the default TRIG value from the conf.bfd file to the top of the bfd script. Now the rule TRIG value does not get overwritten.
  2. The bfd/tmp directory contains files with the last size of the log file in them. The first time you run bfd the bfd/tmp directory gets populated with these files. Since they are empty, containing no size value, the first time you run the bfd program it skips the entire log file. Dam hard to see if your new rule logic is working if the entire log file you think you are processing is skipped! I changed the logic in this line in the tlog script from:
  echo “0 > $BASERUN/$TLOG
  to
  echo “1″ > $BASERUN/$TLOG
  now the first time you run the script it actually processes the log file and you can see if your new rule is working!
  Thank you for this bfd script, please include source documentation with it so the GNU community can contribute to it more easily.
  Thanks again.
  - #32 written by Ryan M.
    about 2 years ago
    
    Reply Quote
    
    1. The TRIG value reloading is intended, it is reset for each rule as it is processed. If you want to override this you can place a TRIG value inside the rule file itself.
    2. On first runs of BFD and for new rules, it is intended by design that BFD begin tracking a log from the point at which the log is currently at. It would, especially on larger servers, cause undue loads for BFD on installation or running of a new rule, process entire logs up to the current time stamp. As such it is a trade off that we have BFD start tracking a log from the point of installation / where the log is currently at when a new rule is run. The changes you have made, for your own needs, are perfectly valid.
#33 written by abubin
about 2 years ago

Reply Quote

i have installed APF+BFD but seems like BFD is not taking effects. I have proftpd running which I tried to login using failed account but BFD doesn’t seems to be blocking it. Any idea?

I already run bfd -s to start it.
- #34 written by abubin
  about 2 years ago
  
  Reply Quote
  
  does the forum still exist? i found out ssh does get blocked by bfd but proftd doesn’t. I checked in proftd settings already to no avail. Not sure where I should get support on this issue…
#35 written by Cygz
about 2 years ago

Reply Quote

I’ve been using bfd for some time on older systems looking to get back into things and of course have been testing it on my new systems. Using Ubuntu 10 it doesn’t look like BFD is seeing anything. I get the idea something’s changed but can’t be too sure where to look. Any thoughts?
#36 written by wookielinux
about 2 years ago

Reply Quote

could someone please show me an example of how to create a rule for trapping 10+ 404 error requests from apache log?

I have been trying to create my own but sed an me dont seem to agree on anything \;)
- #37 written by Russ
  about 2 years ago
  
  Reply Quote
  
  This script may work in your situation. I use it to catch php exploits. Place it in your rules directory as httpd.
  
  # failed logins from a single address before ban
  # uncomment to override conf.bfd trig value
  TRIG=”10″
  
  # file must exist for rule to be active
  REQ=”/var/log/httpd/error_log”
  
  if [ -f "$REQ" ]; then
  LP=”/var/log/httpd/error_log”
  TLOG_TF=”httpd”
  
  # Max logs to process at one time for this rule
  MLOG=1500
  
  ## httpd errors
  ARG_VAL=`$TLOG_PATH $LP $TLOG_TF | grep -iE “File does not exist: /var/www/html/” | sed -e ‘s/::ffff://’ | awk ‘{print$8}’| tr -d ‘[],’ | grep -E ‘[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+’| tail -n $MLOG | sort -n`
  fi
#38 written by Bob
about 2 years ago

Reply Quote

Is there any setting or way to have a banned ip expire over a set period of time?
#39 written by Paul Grant
about 2 years ago

Reply Quote

Hi,
Thanks for the software, both this and LMD are packages that I want to roll out across our entire network.

I’ve added some additional regex for the proftp rule to find incorrect passwords on existing ftp accounts (This may be Plesk specific ) :

ARG_VAL=`$TLOG_PATH $LP $TLOG_TF | sed -e ‘s/::ffff://’ | grep -E ‘[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+’ | tr -d ‘:’ | sed -n -e ‘/proftpd/s/.*USER $.*$ no such user found from $[^ ]*$.*/\2:\1/p’ -e ‘/proftd/s/.*\[$.*$\].*USER $\b[^ ]*$ .*Login failed.*/\2:\1/p’`

This may or may not be helpful to someone else.

Paul.
#40 written by Miha
about 2 years ago

Reply Quote

Hi Ryan..

the cron doesn’t work on ubuntu.. the problem is with mail line.. it should read MAILTO=”"…. Or Maybe it would be better to add >/dev/null 2>&1 at the end of command?
#41 written by Roger
about 2 years ago

Reply Quote

Hello,

Can you help me, please?

What would the command to Deny a range of IP?
For example 92.38.128.0 to 92.38.255.255

Thanks
#42 written by doug
about 2 years ago

Reply Quote

So, running APF + BFD at the same time is over kill right?

I’m actually doing this at the moment, and it seems redundant.

APF sends me:
The following is a summary event for exceeded login failures on server.spanet.net:

SOURCE ADDRESS: 89.122.36.158
TARGET SERVICE: sshd
FAILED LOGINS: 95
EXECUTED COMMAND: /etc/apf/apf -d 89.122.36.158 {bfd.sshd}

SOURCE LOGS FROM SERVICE ‘sshd’ (GMT -0400):

Jul 1 07:54:47 server sshd[13794]: Failed password for root from ::ffff:89.122.36.158 port 57538 ssh2
Jul 1 07:54:47 server sshd[13795]: Failed password for root from ::ffff:89.122.36.158 port 57545 ssh2

BFD sends me (enabled thru WHM/Cpanel):
5 failed login attempts to account root (system) — Large number of attempts from this IP: 89.122.36.158

I have been using your APF server saver for years (installed via ssh). Only recently I decided to enable BFD also (from WHM).

Your thoughts on this would be helpful.
Thank you for your time and dedication to server security!
- #43 written by Ryan M.
  about 2 years ago
  
  Reply Quote
  
  The alerts you think are from APF are actually BFD, APF is a firewall facility, BFD detects brute force attacks and hands them off to APF for filtering. The BFD features in WHM are independent of the rfxn.com BFD project you have installed, WHM BFD though overlaps with rfxn.com BFD in some areas, in general, rfxn.com BFD covers more services for you and as such blankets the server with better protection. So, there should be no real performance impact by leaving rfxn.com BFD and WHM BFD enabled, do not uninstall APF as it is not in any way performing brute force detection, it is simply the firewall facility that allows or denies traffic to the server based on its configuration.
  
  For more details on each of the respective projects please refer to their project pages:
  http://www.rfxn.com/projects/advanced-policy-firewall/
  http://www.rfxn.com/projects/brute-force-detection/
#44 written by Ryan M.
about 2 years ago

Reply Quote

BFD runs as a cronjob so after reboots, it will still execute normally. You should see bans in /var/log/bfd_log assuming things are running properly. You can run bfd -s and see if it is running without errors.

Darin:

=If an IP has been banned, which log will the banned IP be in?

And if I do a container reboot, will this program still be in affect?
#45 written by Darin
about 2 years ago

Reply Quote

=If an IP has been banned, which log will the banned IP be in?

And if I do a container reboot, will this program still be in affect?
#46 written by Darin
about 2 years ago

Reply Quote

If an IP has been banned, which log will the banned IP be in?
- #47 written by Benjamin
  about 2 years ago
  
  Reply Quote
  
  Darin:
  
  If an IP has been banned, which log will the banned IP be in?
  
  BFD itself logs in /var/log/bfd_log
#48 written by Unihost Brasil - Hospedagem confiável de sites
about 3 years ago

Reply Quote

Thank you for this valuable piece of software!

Brute Force Detection

Donation Roll

LMD Malware Updates