Linux Malware Detect

Current Release:
http://www.rfxn.com/downloads/maldetect-current.tar.gz
http://www.rfxn.com/appdocs/README.maldetect
http://www.rfxn.com/appdocs/CHANGELOG.maldetect

Description
Linux Malware Detect (LMD) is a malware scanner for Linux released under the GNU GPLv2 license, that is designed around the threats faced in shared hosted environments. It uses threat data from network edge intrusion detection systems to extract malware that is actively being used in attacks and generates signatures for detection. In addition, threat data is also derived from user submissions with the LMD checkout feature and from malware community resources. The signatures that LMD uses are MD5 file hashes and HEX pattern matches, they are also easily exported to any number of detection tools such as ClamAV.

The driving force behind LMD is that there is currently limited availability of open source/restriction free tools for Linux systems that focus on malware detection and more important that get it right. Many of the AV products that perform malware detection on Linux have a very poor track record of detecting threats, especially those targeted at shared hosted environments.

The threat landscape in shared hosted environments is unique from that of the standard AV products detection suite in that they are detecting primarily OS level trojans, rootkits and traditional file-infecting viruses but missing the ever increasing variety of malware on the user account level which serves as an attack platform.

The commercial products available for malware detection and remediation in multi-user shared environments remains abysmal. An analysis of 8,883 malware hashes, detected by LMD 1.5, against 30 commercial anti-virus and malware products paints a picture of how poorly commercial solutions perform.

DETECTED KNOWN MALWARE: 1951
% AV DETECT (AVG): 58
% AV DETECT (LOW): 10
% AV DETECT (HIGH): 100
UNKNOWN MALWARE: 6931

Using the Team Cymru malware hash registry, we can see that of the 8,883 malware hashes shipping with LMD 1.5, there was 6,931 or 78% of threats that went undetected by 30 commercial anti-virus and malware products. The 1,951 threats that were detected had an average detection rate of 58% with a low and high detection rate of 10% and 100% respectively. There could not be a clearer statement to the need for an open and community driven malware remediation project that focuses on the threat landscape of multi-user shared environments.

Features:
– MD5 file hash detection for quick threat identification
– HEX based pattern matching for identifying threat variants
– statistical analysis component for detection of obfuscated threats (e.g: base64)
– integrated detection of ClamAV to use as scanner engine for improved performance
– integrated signature update feature with -u|–update
– integrated version update feature with -d|–update-ver
– scan-recent option to scan only files that have been added/changed in X days
– scan-all option for full path based scanning
– checkout option to upload suspected malware to rfxn.com for review / hashing
– full reporting system to view current and previous scan results
– quarantine queue that stores threats in a safe fashion with no permissions
– quarantine batching option to quarantine the results of a current or past scans
– quarantine restore option to restore files to original path, owner and perms
– quarantine suspend account option to Cpanel suspend or shell revoke users
– cleaner rules to attempt removal of malware injected strings
– cleaner batching option to attempt cleaning of previous scan reports
– cleaner rules to remove base64 and gzinflate(base64 injected malware
– daily cron based scanning of all changes in last 24h in user homedirs
– daily cron script compatible with stock RH style systems, Cpanel & Ensim
– kernel based inotify real time file scanning of created/modified/moved files
– kernel inotify monitor that can take path data from STDIN or FILE
– kernel inotify monitor convenience feature to monitor system users
– kernel inotify monitor can be restricted to a configurable user html root
– kernel inotify monitor with dynamic sysctl limits for optimal performance
– kernel inotify alerting through daily and/or optional weekly reports
– e-mail alert reporting after every scan execution (manual & daily)
– path, extension and signature based ignore options
– background scanner option for unattended scan operations
– verbose logging & output of all actions

Source Data:
The defining difference with LMD is that it doesn’t just detect malware based on signatures/hashes that someone else generated but rather it is an encompassing project that actively tracks in the wild threats and generates signatures based on those real world threats that are currently circulating.

There are four main sources for malware data that is used to generate LMD signatures:
Network Edge IPS: Through networks managed as part of my day-to-day job, primarily web hosting related, our web servers receive a large amount of daily abuse events, all of which is logged by our network edge IPS. The IPS events are processed to extract malware url’s, decode POST payload and base64/gzip encoded abuse data and ultimately that malware is retrieved, reviewed, classified and then signatures generated as appropriate. The vast majority of LMD signatures have been derived from IPS extracted data.
Community Data: Data is aggregated from multiple community malware websites such as clean-mx and malwaredomainlist then processed to retrieve new malware, review, classify and then generate signatures.
ClamAV: The HEX & MD5 detection signatures from ClamAV are monitored for relevant updates that apply to the target user group of LMD and added to the project as appropriate. To date there has been roughly 400 signatures ported from ClamAV while the LMD project has contributed back to ClamAV by submitting over 1,100 signatures and continues to do so on an ongoing basis.
User Submission: LMD has a checkout feature that allows users to submit suspected malware for review, this has grown into a very popular feature and generates on average about 30-50 submissions per week.

Signature Updates:
The LMD signature are updated typically once per day or more frequently depending on incoming threat data from the LMD checkout feature, IPS malware extraction and other sources. The updating of signatures in LMD installations is performed daily through the default cron.daily script with the –update option, which can be run manually at any time.

An RSS feed is available for tracking malware threat updates: http://www.rfxn.com/api/lmd

Detected Threats:
LMD 1.5 has a total of 10,822 (8,908 MD5 / 1,914) signatures, before any updates. The top 60 threats by prevalence detected by LMD are as follows:

base64.inject.unclassed     perl.ircbot.xscan
bin.dccserv.irsexxy         perl.mailer.yellsoft
bin.fakeproc.Xnuxer         perl.shell.cbLorD
bin.ircbot.nbot             perl.shell.cgitelnet
bin.ircbot.php3             php.cmdshell.c100
bin.ircbot.unclassed        php.cmdshell.c99
bin.pktflood.ABC123         php.cmdshell.cih
bin.pktflood.osf            php.cmdshell.egyspider
bin.trojan.linuxsmalli      php.cmdshell.fx29
c.ircbot.tsunami            php.cmdshell.ItsmYarD
exp.linux.rstb              php.cmdshell.Ketemu
exp.linux.unclassed         php.cmdshell.N3tshell
exp.setuid0.unclassed       php.cmdshell.r57
gzbase64.inject             php.cmdshell.unclassed
html.phishing.auc61         php.defash.buno
html.phishing.hsbc          php.exe.globals
perl.connback.DataCha0s     php.include.remote
perl.connback.N2            php.ircbot.InsideTeam
perl.cpanel.cpwrap          php.ircbot.lolwut
perl.ircbot.atrixteam       php.ircbot.sniper
perl.ircbot.bRuNo           php.ircbot.vj_denie
perl.ircbot.Clx             php.mailer.10hack
perl.ircbot.devil           php.mailer.bombam
perl.ircbot.fx29            php.mailer.PostMan
perl.ircbot.magnum          php.phishing.AliKay
perl.ircbot.oldwolf         php.phishing.mrbrain
perl.ircbot.putr4XtReme     php.phishing.ReZulT
perl.ircbot.rafflesia       php.pktflood.oey
perl.ircbot.UberCracker     php.shell.rc99
perl.ircbot.xdh             php.shell.shellcomm

Real-Time Monitoring:
The inotify monitoring feature is designed to monitor paths/users in real-time for file creation/modify/move operations. This option requires a kernel that supports inotify_watch (CONFIG_INOTIFY) which is found in kernels 2.6.13+ and CentOS/RHEL 5 by default. If you are running CentOS 4 you should consider an inbox upgrade with:
http://www.rfxn.com/upgrade-centos-4-8-to-5-3/

There are three modes that the monitor can be executed with and they relate to what will be monitored, they are USERS|PATHS|FILES.

       e.g: maldet --monitor users
       e.g: maldet --monitor /root/monitor_paths
       e.g: maldet --monitor /home/mike,/home/ashton

The options break down as follows:
USERS: The users option will take the homedirs of all system users that are above inotify_minuid and monitor them. If inotify_webdir is set then the users webdir, if it exists, will only be monitored.
PATHS: A comma spaced list of paths to monitor
FILE: A line spaced file list of paths to monitor

Once you start maldet in monitor mode, it will preprocess the paths based on the option specified followed by starting the inotify process. The starting of the inotify process can be a time consuming task as it needs to setup a monitor hook for every file under the monitored paths. Although the startup process can impact the load temporarily, once the process has started it maintains all of
its resources inside kernel memory and has a very small userspace footprint in memory or cpu usage.

See http://www.rfxn.com/appdocs/README.maldetect for more details on inotify monitoring.

Funding:
Funding for the continued development and research into this and other projects, is solely dependent on public contributions and donations. If this is your first time using this software we ask that you evaluate it and consider a small donation; for those who frequent and are continued users of this and other projects we also ask that you make an occasional small donation to help ensure the future of our public projects.

211 Replies to “Linux Malware Detect”

  1. Just wanted to say thanks. My server was hacked a couple of days ago, and I knew something was up… then I got the bandwidth bill: £180!!! Your program pinpointed exactly what was up and quarantined it.

  2. Good day, Ryan:

    I hope you and your family are doing well.

    We’ve received two separate reports from two different servers showing:

    FILE HIT LIST:
    {HEX}php.exe.globals.383 : [full path to a directory]

    Where it shows scores of directories, but no individual files.

    Should I be concerned?

    How would I further diagnose this issue?

    Thank you.

  3. First off, great tool!!

    Here’s my uname -a

    Linux xxxxxxxx 2.6.32-32-server #62-Ubuntu SMP Wed Apr 20 22:07:43 UTC 2011 x86_64 GNU/Linux

    Couple of bugs I found.

    1. tlog doesn’t like /bin/sh under Ubuntu 10.04, changing to /bin/bash seems to fix that.
    2. Including and using 32bit bins for inotifywatch and its library really doesn’t help those of us using x64 🙁

    Any change in an updated version including some logic to use the system version of inotifywatch?

    1. The tlog issue has been fixed and pushed live, sorry for the oversight.

      As for the inotify binaries, in 1.4.2 I will add x64 binaries along with a simple check to determine system arch and use the appropriate binaries. I will also add a check to see if the system has its own copy of inotifywatch in $PATH and if so use it.

  4. Hi, thanks for this LIFESAVING application! Your excellent work is amazing and so useful to thousands of people, and I truly appreciate it.

    I was just wondering… I dont see options for quiet/verbose. I think this generally standard unix feature would be quite helpful…

    For example, I have a nightly cron with email report, something like this:

    maldet -d
    maldet -u
    maldet -r …

    and the email is so long with unnecessary info.

    I wish there was a way to just have it report the minimal info, without the repeated headers and emails…
    something like: maldet -d –quiet
    or –noheader

    What do you think?

  5. Installed this anti-malware, and I’m happy effect. However, it is difficult to cope with the recent malware injection generating htaccess files. I hope that this will be improved.

  6. hello, nice tool and great work. is there a way to build own hashes, for example i need to find files with binaries or bashes:

    the last server ist hacked by an unsecure apache with root kit spl.sh :

    #!/bin/sh
    umask 0
    LD_AUDIT=libpcprofile.so PCPROFILE_OUTPUT=/etc/ld.so.preload ping
    echo “[+] creating /tmp/getuid.so”
    echo “int getuid(){return 0;}” > /tmp/getuid.c
    gcc -shared /tmp/getuid.c -o /tmp/getuid.so
    echo “/tmp/getuid.so” > /etc/ld.so.preload

    So i want to build an hash to parse all files with #!/bin/ or for binaries:

    file /bin/bash -> search/scan for ELF \d\d-bit

    So which files i have to touch for own hashes ?

    hex.dat
    md5.dat
    rfxn.hdb
    rfxn.ndb

    so for example want to find files with “#!/bin”

    what i have to do ?

    md5(#!/bin) -> touch md5.dat and/or rfxn.db ->

    0b4962363758288f8f7e0d9cdb413d92:88:{MD5}bash.inject.unclassed.1

    0b4962363758288f8f7e0d9cdb413d92:{MD5}bash.inject.unclassed.1

    do we need both hex and md5 for this ? thanks for any help with own definitions cause i think not everyone wants to parse #!/bin

  7. never mind this, /tmp actually had 40k files.
    You should however add a feature that allows me to scan only php files in all folders recursively
    something like
    maldet -a /home/user/public_html/?/?.php

    that command only digs in one level deep.


    Binoy:

    I am trying to scan a user directory and it reports there are 42500 files to scan, which will obviously take a long time
    The number of files in user directory is only close to 5000 (using find . -type f )
    Nm this

    The number of files in /var/tmp is 1800
    /tmp is symlink of /var/tmp
    where is maldet getting 42500 files from ? and how to ignore them.
    Can you provide a full list of all the directories that maldetect automatically add.
    It would be helpful if maldet ignores these directories if they were added to ignore_paths


    Binoy:

    I am trying to scan a user directory and it reports there are 42500 files to scan, which will obviously take a long time
    The number of files in user directory is only close to 5000 (using find . -type f )
    The number of files in /var/tmp is 1800
    /tmp is symlink of /var/tmp
    where is maldet getting 42500 files from ? and how to ignore them.
    Can you provide a full list of all the directories that maldetect automatically add.
    It would be helpful if maldet ignores these directories if they were added to ignore_paths

  8. I am trying to scan a user directory and it reports there are 42500 files to scan, which will obviously take a long time

    The number of files in user directory is only close to 5000 (using find . -type f )
    The number of files in /var/tmp is 1800
    /tmp is symlink of /var/tmp

    where is maldet getting 42500 files from ? and how to ignore them.
    Can you provide a full list of all the directories that maldetect automatically add.
    It would be helpful if maldet ignores these directories if they were added to ignore_paths

  9. This looks like an excellant little tool! All works well with -a, malicious files are picked up and see these in the report. Where using monitor and malicious content is uploaded I’m seeing the relevant inotify in the event_log but seemingly is not detected as a bad file (Not logged nor do I get an email about it when I run the cron). Does anyone have any suggestions? CentOS 5.7, CloudLinux kernel.

  10. Hi,
    is there any chance to protect the /etc/cron.daily/maldet from changing by Updateprocess? I’m on Debian and hafe to use an other tool for the clean the tmps and other Paths for my home Users.

    PS: The function ‘ignore_sigs’ still wont work 🙁

  11. I have put my server scan using the command
    maldet –scan-all /home?/?/public_html &

    & click on close button, now how to check how may files scanned ??? & what maldet is currently doing ??

  12. Hello,

    how could i get the ignore_sigs file working?
    The hit is:

    I tested the entries

    gzbase64.inject.unclassed
    gzbase64.inject.unclassed.14
    {HEX}gzbase64.inject.unclassed.14
    and
    {HEX}gzbase64.inject.unclassed.14:0:*:677a696e666c617465286261736536345f6465636f646528

    But all wont ignored.

    Thank you!

Leave a Reply

Your email address will not be published. Required fields are marked *