Brute Force Detection

Current Release:
http://www.rfxn.com/downloads/bfd-current.tar.gz
http://www.rfxn.com/appdocs/README.bfd
http://www.rfxn.com/appdocs/CHANGELOG.bfd

Description
BFD is a modular shell script for parsing application logs and checking for authentication failures. It does this using a rules system where application-specific options are stored including regular expressions for each unique auth format. The regular expressions are parsed against logs using the ‘sed’ tool (stream editor) which allows for excellent performance in all environments. In addition to the benefits of parsing logs in a single stream with sed, BFD also uses a log tracking system so logs are only parsed from the point which they were last read. This greatly assists in extending the performance of BFD even further as we are not constantly reading the same log data. The log tracking system is compatible with syslog/logrotate style log rotations which allows it to detect when rotations have happened and grab log tails from both the new log file and the rotated log file.

You can leverage BFD to block attackers using any number of tools such as APF, Shorewall, raw iptables, ip route or execute any custom command. There is also a fully customizable e-mail alerting system with an e-mail template that is well suited for everyday use or you can open it up and modify it. The attacker tracking in BFD is handled using simple flat text files that are size-controlled to prevent space constraints over time, ideal for diskless devices. There is also an attack pool where trending data is stored on all hosts that have been blocked including which rule the block was triggered by.

In the execution process, there is simply a cron job that executes BFD once every 3 minutes by default. The cronjob can be run more frequently for those that desire it and doing so will not cause any performance issues (no less than once a minute). Although cron execution does not permit BFD to act in real time, the log tracking system ensures it never misses a beat in authentication failures. Further, using cron provides a reliable framework for consistent execution of BFD in a very simplified fashion across all *nix platforms.

Funding:
Funding for the continued development and research into this and other projects is solely dependent on public contributions and donations. If this is your first time using this software we ask that you evaluate it and consider a small donation; for those who frequent and are continued users of this and other projects we also ask that you make an occasional small donation to help ensure the future of our public projects.

100 Replies to “Brute Force Detection”

  1. also I’d like to know if the number of allowed failures is related to a certain time frame, i.e. 10 failures / 10 minutes?

    # how many failure events must an address have before being blocked?
    # you can override this on a per rule basis in /usr/local/bfd/rules/
    TRIG=”10″

    1. The max failures (TRIG) is per IP address and is an incrementing value across each BFD execution, so even if you have BFD run every 1 minute or 10 minutes, it will track the authentication failures for an IP address in total and when it reaches TRIG failures, it gets banned.

  2. I am trying to use BFD instead of fail2ban but I am unsure about it because I can’t seem to find a setting as to how long a misbehaving hostname is being blocked.
    And does BFD act similar to fail2ban and unblocks a certain hostname after a specific amount of time?

  3. Hello Ryan,

    Please consider to remove the filter command for the content of the BAN_COMMAND (line 180 in the bfd version 1.5): BAN_COMMAIND=`echo $BAN_COMMAND | tr -d ‘\\&;|’`

    We are using the following BAN_COMMAND and we have to delete the filter from the bfd script:

    BAN_COMMAND=”(/sbin/iptables -n -L | grep DROP | grep $ATTACK_HOST) || /sbin/iptables -I INPUT -s $ATTACK_HOST -j DROP”

    Off course we could put there an external script but we are proffering to not add another layer of bash commands.

    BR,
    Ioan

  4. Hi,

    I’m very happy with your AFP and BFD tools, but I have a question about configuring BFD: by default, conf.bfd has:

    BAN_COMMAND=”/etc/apf/apf -d $ATTACK_HOST {bfd.$MOD}”

    This adds the hostname of the the attacker to the ban list, but is there a way to add their IP address instead?

  5. Found a small bug in BFD 1.4. Somewhere at line 153 is listed:

    if [ “$ATTACK_COUNT” -gt “$TRIG” ] || [ “$ATTACK_COUNT” -eq “$TRIG” ] && [ “$HOST_IGNORE” == “0” ]; then

    However, when $HOST_IGNORE is null, $ATTACK_COUNT is not defined at all. This will result into a script error:

    [: : integer expression expected

    Defining ATTACK_COUNT somewhere at the start of the script will fix this:
    ATTACK_COUNT=0

    Not sure how the working of BFD is affected by this error…


  6. Roland:

    great script, however i’m getting 1000′s of these in my exim logs and it doesn’t ban them:
    2011-06-28 05:47:38 login authenticator failed for (ylmf-pc) [113.65.143.13]: 535 Incorrect authentication data (set_id=web)
    does anyone have a rule for exim that will ban these as well?
    thanks in advance
    roland

    I am using the following rule for this:

    # failed logins from a single address before ban
    # uncomment to override conf.bfd trig value
    TRIG=”50″

    # file must exist for rule to be active
    REQ=”/usr/sbin/exim”

    if [ -f “$REQ” ]; then
    LP=”/var/log/exim/mainlog”
    TLOG_TF=”exim1″

    ## EXIM attacks
    ARG_VAL=`$TLOG_PATH $LP $TLOG_TF | grep -E “login authenticator failed for” | grep -E ‘[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+’ | tr -d ‘[]’ | sed -n -e ‘s/.*login authenticator failed for \([^ ]*\) .* \([\.0-9]*\): 535 Incorrect authentication data.*/\2:\1/p’`
    fi

  7. Hello,

    Thank you for this great project
    I use it but I find that the system detect the attack but it doesn’t really apply the ban command when it run by the cron job and when I run bfd -q or bfd -s it works and apply the command. by the way I change the ban command to blackhole the IP “”BAN_COMMAND=”ip route add blackhole $ATTACK_HOST”
    Also there is a problem in the system that if the IP in the ban.list file the system ignore any attack from the IP, this happen when the system detect the ip but the command didn’t applied

    Also I see that the IPs in pan.list file removed after some time, how does this configured and what’s the default time.

    Thanks


    1. Biruny:

      Hello,
      Thank you for this great project
      I use it but I find that the system detect the attack but it doesn’t really apply the ban command when it run by the cron job and when I run bfd -q or bfd -s it works and apply the command. by the way I change the ban command to blackhole the IP “”BAN_COMMAND=”ip route add blackhole $ATTACK_HOST”
      Also there is a problem in the system that if the IP in the ban.list file the system ignore any attack from the IP, this happen when the system detect the ip but the command didn’t applied
      Also I see that the IPs in pan.list file removed after some time, how does this configured and what’s the default time.
      Thanks

      Hello,

      any update about these comments ??
      dose any one know the problems ?

  8. I have made a few scripts for some very-specific services. You shouldn’t have to be using Debian, but they are confirmed working in Debian with all the latest packages. For SASL-based authentication failures in Postfix, try the following:

    — BEGIN /usr/local/bfd/rules/postfix —

    # failed logins from a single address before ban
    # uncomment to override conf.bfd trig value
    #TRIG=”50″

    # file must exist for rule to be active
    REQ=”/usr/sbin/postfix”

    if [ -f “$REQ” ]; then
    LP=”/var/log/mail.log”
    TLOG_TF=”postfix”

    ## Postfix dictionary attacks
    ARG_VAL=`$TLOG_PATH $LP $TLOG_TF | grep -iwE “SASL LOGIN authentication failed:|SASL PLAIN authentication failed:” | grep -E ‘[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+’ | cut -d ‘ ‘ -f 7 | tr -d ‘[a-z][A-Z]\[\]\:’`
    fi

    — END —

    Also, if you have your postfix daemon properly configured to block invalid HELO tries, the user actually won’t even make it as far as authentication when faking HELOs, but you will get lots of annoying log entries about it. Here’s a script called postfix2 to also block many failed HELO attempts:

    — BEGIN /usr/local/bfd/rules/postfix2 —

    # failed logins from a single address before ban
    # uncomment to override conf.bfd trig value
    #TRIG=”50″

    # file must exist for rule to be active
    REQ=”/usr/sbin/postfix”

    if [ -f “$REQ” ]; then
    LP=”/var/log/mail.log”
    TLOG_TF=”postfix2″

    ## Postfix failed HELOs
    ARG_VAL=`$TLOG_PATH $LP $TLOG_TF | grep -iwE “lost connection after RCPT from” | grep -E ‘[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+’ | awk -F \[ ‘{ print $3 }’| tr -d ‘[]’`
    fi

    — END —

    Finally, for vsftpd failures, try this script that I borrowed from someone else and modified slightly:

    — BEGIN /usr/local/bfd/rules/vsftpd —

    REQ=”/usr/sbin/vsftpd”
    if [ -f “$REQ” ]; then
    LP=”/var/log/vsftpd.log”
    TLOG_TF=”vsftpd”
    TRIG=”6″

    ## VSFTPD
    ARG_VAL=`$TLOG_PATH $LP $TLOG_TF | grep -w ‘FAIL LOGIN:’ | tr ‘[]’ ‘ ‘ | tr -d ‘()’ | awk ‘{print$12″ “$8}’ | tr -d ‘:’ | tr -d ‘”‘ | awk ‘{print$1”:”$2}’ | grep -E ‘[0-9]+’`
    fi

    — END —

    Hope those help someone.

    1. Found an error in ‘postfix’ that would cause invalid addresses when the hostname had numbers (kept the numbers and dots so it usually looked something like 207..123.45.67.89. Here is the new and improved postfix script, which works with numbered hostnames:

      — BEGIN /usr/local/bfd/rules/postfix —

      # failed logins from a single address before ban
      # uncomment to override conf.bfd trig value
      #TRIG=”50″

      # file must exist for rule to be active
      REQ=”/usr/sbin/postfix”

      if [ -f “$REQ” ]; then
      LP=”/var/log/mail.log”
      TLOG_TF=”postfix”

      ## Postfix dictionary attacks
      ARG_VAL=`$TLOG_PATH $LP $TLOG_TF | grep -iwE “SASL LOGIN authentication failed:|SASL PLAIN authentication failed:” | grep -E ‘[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+’ | awk -F ‘\[‘ ‘{ print $3 }’ | tr -d ‘[a-z][A-Z]\[\]\:’`
      fi

      — END —

      Tested on Debian (sid)’s postfix, and a few others.

      1. Thank you for your contribution, added to 1.5 release that is now live, credit included in CHANGELOG. I removed the escape on the -F ‘\[‘ as it was being treated as just ‘[‘ anyways and awk would throw a warning about it under RHEL awk versions.

  9. hi remco

    this works for me for dovecot you can give it a try:

    REQ=”/usr/sbin/dovecot”
    if [ -f “$REQ” ]; then
    LP=”/var/log/maillog”
    TLOG_TF=”pop3″
    TRIG=”8″

    ## pop3
    ARG_VAL=`$TLOG_PATH $LP $TLOG_TF |grep dovecot |grep -w “failed” |grep auth |grep -o -E ‘[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+’`
    fi

  10. great script, however i’m getting 1000’s of these in my exim logs and it doesn’t ban them:
    2011-06-28 05:47:38 login authenticator failed for (ylmf-pc) [113.65.143.13]: 535 Incorrect authentication data (set_id=web)

    does anyone have a rule for exim that will ban these as well?

    thanks in advance

    roland

  11. Hi Ryan,

    I’m wondering if it would be possible to use bfd with the uw-imap daemon. My server uses this as part of the Parallels Pro (formerly Ensim) control panel and we’re getting bombarded with imap floods.

    Thanks for a great set of products and for your ongoing work.

    -Ray

  12. I am a major fan of rfxn.

    BFD actually saved my small company. I use APF & BFD and without it, I would be out of business because I can not afford the expertise that rfxn possesses.

    Thank you to the rfxn team! My donation en-route.

  13. Thanks Ryan. I’ll give Ash a try, then.

    Fail2Ban won’t do on this appliance because it’s written in Python and there isn’t enough RAM to fit.

  14. Hello

    I’m looking for a good way to ban IP’s trying to register with my Asterisk server illegally, and was thinking of installing BFD to work with iptables.

    Problem is, it’s an appliance running uClinux and Busybox, which provides the following shells: ash, hush, lash, and msh.

    Will BFD work with those shells?

    Thank you.

      1. I know this is an old entry but did anyone ever get rules put together for this? I would be very interested in see them or helping make them. I am currently running the FreePBX distro with Asterisk Ver. 1.8.12.0.

  15. Hello,

    Running with bfd 1.4 and found a “minor bug” in bfd script:
    ATTACK_COUNT=$[RET_COUNT+ATTACK_COUNT] should be
    ATTACK_COUNT=RET_COUNT (line #150)
    otherwise the attack is considered valid (ATTACK_COUNT -ge TRIG) as the ATTACK_COUNT lines have been already written into the track.attack file (line #136)

    Best regards,
    Ioan

  16. Hi, I am trying to write a rule for dovecot.
    I created this:
    ARG_VAL=`$TLOG_PATH $LP $TLOG_TF |grep ‘mail dovecot: pop3-login: Disconnected:’ |awk ‘{print$10}’ | grep -o \
    -E ‘[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+’`

    When I run it against the maillog, the hackers IP is listed over and over, but when I create the above rule and I run /usr/local/sbin/bfd -s
    the rule runs but it does not capture the hacker’s IP. Could you please explain what the rule should return or how to write a new rule or just point me to a doc where you have described what the rule should be returning to the bfd script?
    Thank you

    1. Ok, I was able to figure out the rules must return the IP:user to the bfd script.
      Example:
      50.7.246.130:admin

      Knowing this I can at last play with creating new rules. I found two problems with the bfd script while creating my new rule.
      1. the TRIG variable is reset to the default value from the conf.bfd file after the first time the rule’s TRIG value is met because the conf.bfd is reloaded. To fix this I moved the default TRIG value from the conf.bfd file to the top of the bfd script. Now the rule TRIG value does not get overwritten.
      2. The bfd/tmp directory contains files with the last size of the log file in them. The first time you run bfd the bfd/tmp directory gets populated with these files. Since they are empty, containing no size value, the first time you run the bfd program it skips the entire log file. Dam hard to see if your new rule logic is working if the entire log file you think you are processing is skipped! I changed the logic in this line in the tlog script from:
      echo “0 > $BASERUN/$TLOG
      to
      echo “1” > $BASERUN/$TLOG
      now the first time you run the script it actually processes the log file and you can see if your new rule is working!
      Thank you for this bfd script, please include source documentation with it so the GNU community can contribute to it more easily.
      Thanks again.

      1. 1. The TRIG value reloading is intended, it is reset for each rule as it is processed. If you want to override this you can place a TRIG value inside the rule file itself.
        2. On first runs of BFD and for new rules, it is intended by design that BFD begin tracking a log from the point at which the log is currently at. It would, especially on larger servers, cause undue loads for BFD on installation or running of a new rule, process entire logs up to the current time stamp. As such it is a trade off that we have BFD start tracking a log from the point of installation / where the log is currently at when a new rule is run. The changes you have made, for your own needs, are perfectly valid.

  17. i have installed APF+BFD but seems like BFD is not taking effects. I have proftpd running which I tried to login using failed account but BFD doesn’t seems to be blocking it. Any idea?

    I already run bfd -s to start it.

    1. does the forum still exist? i found out ssh does get blocked by bfd but proftd doesn’t. I checked in proftd settings already to no avail. Not sure where I should get support on this issue…

  18. I’ve been using bfd for some time on older systems looking to get back into things and of course have been testing it on my new systems. Using Ubuntu 10 it doesn’t look like BFD is seeing anything. I get the idea something’s changed but can’t be too sure where to look. Any thoughts?

  19. could someone please show me an example of how to create a rule for trapping 10+ 404 error requests from apache log?

    I have been trying to create my own but sed an me dont seem to agree on anything \;)

    1. This script may work in your situation. I use it to catch php exploits. Place it in your rules directory as httpd.

      # failed logins from a single address before ban
      # uncomment to override conf.bfd trig value
      TRIG=”10″

      # file must exist for rule to be active
      REQ=”/var/log/httpd/error_log”

      if [ -f “$REQ” ]; then
      LP=”/var/log/httpd/error_log”
      TLOG_TF=”httpd”

      # Max logs to process at one time for this rule
      MLOG=1500

      ## httpd errors
      ARG_VAL=`$TLOG_PATH $LP $TLOG_TF | grep -iE “File does not exist: /var/www/html/” | sed -e ‘s/::ffff://’ | awk ‘{print$8}’| tr -d ‘[],’ | grep -E ‘[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+’| tail -n $MLOG | sort -n`
      fi

  20. Hi,
    Thanks for the software, both this and LMD are packages that I want to roll out across our entire network.

    I’ve added some additional regex for the proftp rule to find incorrect passwords on existing ftp accounts (This may be Plesk specific ) :

    ARG_VAL=`$TLOG_PATH $LP $TLOG_TF | sed -e ‘s/::ffff://’ | grep -E ‘[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+’ | tr -d ‘:’ | sed -n -e ‘/proftpd/s/.*USER \(.*\) no such user found from \([^ ]*\).*/\2:\1/p’ -e ‘/proftd/s/.*\[\(.*\)\].*USER \(\b[^ ]*\) .*Login failed.*/\2:\1/p’`

    This may or may not be helpful to someone else.

    Paul.

  21. Hi Ryan..

    the cron doesn’t work on ubuntu.. the problem is with mail line.. it should read MAILTO=””…. Or Maybe it would be better to add >/dev/null 2>&1 at the end of command?

  22. Hello,

    Can you help me, please?

    What would the command to Deny a range of IP?
    For example 92.38.128.0 to 92.38.255.255

    Thanks

  23. So, running APF + BFD at the same time is over kill right?

    I’m actually doing this at the moment, and it seems redundant.

    APF sends me:
    The following is a summary event for exceeded login failures on server.spanet.net:

    SOURCE ADDRESS: 89.122.36.158
    TARGET SERVICE: sshd
    FAILED LOGINS: 95
    EXECUTED COMMAND: /etc/apf/apf -d 89.122.36.158 {bfd.sshd}

    SOURCE LOGS FROM SERVICE ‘sshd’ (GMT -0400):

    Jul 1 07:54:47 server sshd[13794]: Failed password for root from ::ffff:89.122.36.158 port 57538 ssh2
    Jul 1 07:54:47 server sshd[13795]: Failed password for root from ::ffff:89.122.36.158 port 57545 ssh2

    BFD sends me (enabled thru WHM/Cpanel):
    5 failed login attempts to account root (system) — Large number of attempts from this IP: 89.122.36.158

    I have been using your APF server saver for years (installed via ssh). Only recently I decided to enable BFD also (from WHM).

    Your thoughts on this would be helpful.
    Thank you for your time and dedication to server security!

    1. The alerts you think are from APF are actually BFD, APF is a firewall facility, BFD detects brute force attacks and hands them off to APF for filtering. The BFD features in WHM are independent of the rfxn.com BFD project you have installed, WHM BFD though overlaps with rfxn.com BFD in some areas, in general, rfxn.com BFD covers more services for you and as such blankets the server with better protection. So, there should be no real performance impact by leaving rfxn.com BFD and WHM BFD enabled, do not uninstall APF as it is not in any way performing brute force detection, it is simply the firewall facility that allows or denies traffic to the server based on its configuration.

      For more details on each of the respective projects please refer to their project pages:
      http://www.rfxn.com/projects/advanced-policy-firewall/
      http://www.rfxn.com/projects/brute-force-detection/

  24. BFD runs as a cronjob so after reboots, it will still execute normally. You should see bans in /var/log/bfd_log assuming things are running properly. You can run bfd -s and see if it is running without errors.


    Darin:

    =If an IP has been banned, which log will the banned IP be in?

    And if I do a container reboot, will this program still be in affect?

Leave a Reply

Your email address will not be published. Required fields are marked *