BFD 1.4: Important Security Fix

Today I have put up a new release of BFD, version 1.4, that addresses an unsanitized variable issue that is used on the command line. This is a serious issue and should be treated as such, if you currently have BFD installed I would encourage you to update it immediately, the install.sh script in the BFD package will retain all your options and tracking data so the update process is painless.

Current Release:
http://www.rfxn.com/downloads/bfd-current.tar.gz

Change Log:
[Fix] properly sanitized vars passed to the command line
[Fix] ignore.hosts is now updated with system addresses on each bfd run
[Note] thanks to [email protected] for invaluable input and pointers

wget http://www.rfxn.com/downloads/bfd-current.tar.gz
tar xvfz bfd-current.tar.gz
cd bfd-1.4/
./install.sh

Although this issue has many mitigating factors that lessen the severity of the potential impact it is nevertheless very serious and best to opt on the side of caution. I need to extend a special thanks to Jeff Petersen of webhostsecurity.com for identifying this issue in a very professional fashion and offering technical input.

(ATF) Aggregate Threat Feed

aggregate feed sexyness
ATF sexyness

For my first post back into things in awhile (a long while), I thought I would introduce everyone to the sexyness that i’ve called the Aggregate Threat Feed or ATF for short. This feed is derived from threat data at work, namely our network edge IPS (a custom snort implementation, another post on that later) and aggregated firewall data from 250+ servers, mostly being brute force/invasive scan attack addresses.

There really is nothing terribly fancy about this, the data is presented in a drop list format that is updated every 15 minutes with an optional variable to adjust the amount of addresses returned:

http://asonoc.com/api/atf.php?top=50 (defaults at 100)

The entries in the list are sorted on the database side by highest event count first, you can optionally view the source and event count entries in the list but this is considered strictly for review purposes (it wouldn’t be of much other use). Take note that the maximum value for ‘top’ is 300.

http://asonoc.com/api/atf.php?top=300&fmt=list

The review data looks something like follows:

IP | SOURCE | EVENTS
———————-
187.45.224.5 ips 227
92.48.206.91 ips 202
69.13.196.47 fw 176
210.17.251.159 ips 130
83.170.110.194 fw 125

This is pretty basic to understand, the distinction to note however is that event numbers for IPS source data can be 50 events against 1 or 20 servers whereas the event count for fw sourced data typically reflect unique servers. So an address sourced from fw data with 200 events, actually hit 200 servers.

The next release of APF due in the coming months, will feature many changes and among them will be the inclusion of ATF as part of the new feed subscription feature. Further, users will have the option to enable reporting to the rfxn.com server that allows your own block data to be included in the ATF database. As more installations opt-in on this feature it will allow data aggregation to reflect a more global threat landscape that truly represents the users of APF (currently active installations based on those fetching the rfxn.com reserved.networks list daily: 46,921).

Also on the agenda is a simple ATF landing page that presents statistical data and some fancy graphs/charts (probably use google api cause im lazy like that), that will allow users to better visualize threats included in the feed and details on the actual events that caused an address to end up in it (snort events, firewall triggers etc..).

EOF

Linux Malware Detectection

[ UPDATE: Linux Malware Detect has been released ]
I have the last few weeks been working on a new project for malware detection on Linux web servers, it is already at a pre-release version in use at work and it has shown phenomenal promise.

Right to it, some background… On a daily basis the network I manage receives a large number of attacks, most of these are web based abuses against common web application vulnerabilities which inject/upload to servers an array of malware such as phishing content, defacement tools, exploits for privilege escalation and irc c&c bots. All these actions are typically logged and recorded by our network edge snort setup which got me to thinking if we started to catalog some of the injected malware, I could hash it and then detect it on servers.

Now, some might be thinking – “network edge IDS? why not convert it to IPS and stop the attacks right away?” – though this is something I am actually in the process of doing, there is a much larger problem and that is content encoding. Allot of malware attacks are coming in these days in base64 and gzip encoded data payloads which snort or any other IDS/IPS products for that matter are currently NOT capable of decoding without use of fancy transparent proxy setups that are out of the scope of standard network edge intrusion detection/prevention.

So, this brings us to a host based solution for malware detection which as it turns out is not so easy as there is no simple sites that actually track malware specifically targeting web applications and the ones that do exist focus primarily on Windows based malware; utterly useless. To address this short coming, what I have done is essentially written a set of tools that extracts from specific ids events the payload data of attacks (decodes if needed) and saves/downloads the content attackers are trying to inject. This data is then processed for false positives by me every couple of days followed by the creation of md5 hashed definitions of the malware for the detection tool. The hashes are compiled in two methods, the first is straight md5 hashes of the data and the second are hashes of “chunked” elements of the data in specific increments and formats as so to detect commonly occurring malware code in otherwise unique files and content types.

The scanner portion of the malware detection tool comes in 3 varieties, the first is a standard “scan all” feature which scans an entire defined path, the second is a “scan recent” feature that can scan a path for content created in the last X days (i.e: /home/*/public_html content created in the last 7 days) and the third is a real time monitoring service component that uses Linux inotify() kernel feature to detect real time file create/move/modify operations and scan content immediately as it is created under user web paths (default /home[2]/user/public_html).

The malware hit management is a very simple anti-virus like quarantine system that moves offending files to ‘INSTALL_PATH/quarantine/’ and logs the exact source path and destination file name in quarantine locker in case you need to restore any data due to false positives (though this should never happen since we are using hashed detection). In addition, the quarantine function can optionally search the process table for running tasks that contain the file name of the offending malware and kicks off a kill -9 against it.

The event management is handled in two ways, for manual user invoked scans from cron/command line, emails are directly dispatched with the scan results including quarantine details – nothing really fancy here. The monitor component that uses inotify() on the other hand, has the potential to generate allot of quarantine events in rapid succession so a standard email out on every hit isn’t appropriate. Instead, we have a daily cron job that runs an internal option in the malware detect tool to read ONLY new lines from a quarantine hit list and dispatch a daily event summary if any quarantine hits are found. Since we are only reading new lines from the hit list, we avoid repetitive daily alerts for events we already know about and retain the hit list as an “all-time” hit list that can later be used to derive trending data / phone home features for global trending.

Finally, the project also contains an internal update function to check for new hashes and runs in the daily cron task in addition to a simple check feature that determines if inotify() based monitoring is running, if it is not then it kicks off against /home[2]/user/public_html a scan for content created in the last 48h.

Snorting the Web Farm

Here are some rules for you snort freaks to chew on that I have found useful in web heavy environments.

alert tcp $HTTP_SERVERS $HTTP_PORTS -> any any (msg:"ET ATTACK RESPONSE x2300 phpshell detected"; content:"Locus7Shell"; nocase; classtype:web-application-activity; reference:url,www.rfxn.com; sid:300010; rev:1;)
alert tcp $HTTP_SERVERS $HTTP_PORTS -> any any (msg:"ET ATTACK RESPONSE RFI Scanner detected"; content:"RFI Scanner"; classtype:web-application-activity; reference:url,www.rfxn.com; sid:300020; rev:2;)
alert tcp $HTTP_SERVERS $HTTP_PORTS -> any any (msg:"ET ATTACK RESPONSE lila.jpg phpshell detected"; content:"CMD PHP"; classtype:web-application-activity; reference:url,www.rfxn.com; sid:300030; rev:2;)
alert tcp $HTTP_SERVERS $HTTP_PORTS -> any any (msg:"ET ATTACK RESPONSE ALBANIA id.php detected"; content:"UNITED ALBANIANS aka ALBOSS PARADISE"; classtype:web-application-activity; reference:url,www.rfxn.com; sid:300040; rev:2;)

alert tcp $HTTP_SERVERS $HTTP_PORTS -> any any (msg:"ET ATTACK RESPONSE Data Cha0s Backdoor"; content:"Data Cha0s Connect Back Backdoor"; classtype:web-application-activity; reference:url,www.rfxn.com; sid:300050; rev:1;)
alert tcp $HTTP_SERVERS $HTTP_PORTS -> any any (msg:"ET ATTACK RESPONSE Data Cha0s Backdoor"; content:"Spawning Shell"; classtype:web-application-activity; reference:url,www.rfxn.com; sid:300051; rev:1;)

alert tcp $HTTP_SERVERS $HTTP_PORTS -> any any (msg:"ET ATTACK RESPONSE c99 phpshell variant"; content:"c999shell v."; classtype:web-application-activity; reference:url,www.rfxn.com; sid:300060; rev:1;)
alert tcp $HTTP_SERVERS $HTTP_PORTS -> any any (msg:"ET ATTACK RESPONSE c99 phpshell variant"; content:"C99 Modified"; classtype:web-application-activity; reference:url,www.rfxn.com; sid:300061; rev:1;)
alert tcp $HTTP_SERVERS $HTTP_PORTS -> any any (msg:"ET ATTACK RESPONSE c99 phpshell variant"; content:"RootShell Security Group"; classtype:web-application-activity; reference:url,www.rfxn.com; sid:300062; rev:1;)
alert tcp $HTTP_SERVERS $HTTP_PORTS -> any any (msg:"ET ATTACK RESPONSE c99 phpshell variant"; content:"Modded by Shadow & Preddy"; classtype:web-application-activity; reference:url,www.rfxn.com; sid:300063; rev:1;)

alert tcp $HTTP_SERVERS $HTTP_PORTS -> any any (msg:"ET ATTACK RESPONSE js insert redirect"; content:"META HTTP-EQUIV='Refresh' CONTENT='0"; nocase; content:"text/javascript"; nocase; pcre:"/daobrains\.info|engaolika\.info|globalsecurityscans\.com|goscanpark\.com|goscansome\.com|goscansoon\.com|goslimscan\.com|neborin\.info|safetyshareonline\.com|securityexamineonline\.com/i"; classtype:web-application-activity; reference:url,blog.unmaskparasites.com/2009/07/23/goscanpark-13-facts-about-malicious-server-wide-meta-redirects/; sid:300070; rev:3;)

alert tcp $HTTP_SERVERS $HTTP_PORTS -> any any (msg:"ET ATTACK RESPONSE DarkMailer"; uricontent:"cgi"; nocase; uricontent:"m="; nocase;  pcre:"/dark\.cgi|dm\.cgi|da\.cgi/iU"; classtype: web-application-attack; sid: 300080; rev:1;)
alert tcp $HTTP_SERVERS $HTTP_PORTS -> any any (msg:"ET ATTACK RESPONSE DarkMailer"; content:"yellsoft"; nocase; content:"DirectMailer v"; nocase; classtype: web-application-attack; sid: 300081; rev:1;)

alert tcp $HTTP_SERVERS $HTTP_PORTS -> any any (msg:"ET ATTACK RESPONSE r57 phpshell footer detected"; content:"http-shell by RST/GHC"; classtype:web-application-activity; reference:url,www.pestpatrol.com/spywarecenter/pest.aspx?id=453096755; sid:2003535; rev:4;)
alert tcp any any -> $HTTP_SERVERS $HTTP_PORTS (msg:"ET ATTACK RESPONSE r57 phpshell source being uploaded"; content:"by 1dt.w0lf"; content:"GHC http"; distance:0; classtype:web-application-activity; reference:url,www.pestpatrol.com/spywarecenter/pest.aspx?id=453096755; sid:2003536; rev:5;)

alert tcp any any -> any any (msg:"ET ATTACK RESPONSE devil_ shellbot: conback request"; content:"Conectando-se em"; classtype:attempted-admin; reference:url,www.rfxn.com; sid:300090; rev:1;)
alert tcp any any -> any any (msg:"ET ATTACK RESPONSE devil_ shellbot: udp flood"; content:"pacotando"; content:"temp"; classtype:attempted-dos; reference:url,www.rfxn.com; sid:300091; rev:1;)
alert tcp any any -> any any (msg:"ET ATTACK RESPONSE devil_ shellbot: udp flood"; content:"de envio"; content:"Total pacotes"; content:"Total bytes"; content:"Tempo"; classtype:attempted-dos; reference:url,www.rfxn.com; sid:300092; rev:1;)

alert tcp any any -> any any (msg:"ET ATTACK RESPONSE /bin/sh shell spawn"; content:"no job control in this shell"; classtype:attempted-admin; reference:url,www.rfxn.com; sid:300100; rev:1;)

BOGON Filtering, Update It

One of the features used by APF to prevent address spoofing is that it filters reserved IP address space, also known as BOGON filtering. This is an otherwise very reliable method to keep out random unallocated spoofed addresses from injecting traffic towards your server, assuming, of course, the list is updated regularly.

We decided a few months ago that we would disable by default all remote features in APF, including the auto-updating of the reserved networks file (BOGON filter list). However, this was done with one ill-fated consequence… That if you do not turn on the update feature yourself you may inadvertently start filtering newly allocated address space. This was a change that in hindsight was probably not the best of ideas and as such has now been turned back on by default in the current APF release.

That is then the purpose of this post, every feature even in its best intentions has the capacity when it goes unmanaged to cause more harm than good and that is exactly what the BOGON filtering will do over time if left enabled and it goes unmanaged. The quick fix to this issue is that you enable the reserved network update feature or disable the use of the reserved network filtering in conf.apf altogether; below are details on conducting either task:

1) This change will make APF download and update to the latest reserved network file daily at 4 am (recommended):
edit /etc/apf/conf.apf and set the option DLIST_RESERVED=”1″
restart APF with /etc/init.d/apf restart

OR

2) This change will disable the use of reserved network filtering from APF altogether (not recommended!):
edit /etc/apf/conf.apf and set the option BLK_RESNET=”0″
restart APF with /etc/init.d/apf restart

I hope this assists anyone who is experiencing random connection issues from clients or to better decide if this feature is right for you. The BOGON list APF uses is maintained at http://www.rfxn.com/downloads/reserved.networks , which is updated twice daily with the latest data from ARIN.