Linux Malware Detect

Current Release:
http://www.rfxn.com/downloads/maldetect-current.tar.gz
http://www.rfxn.com/appdocs/README.maldetect
http://www.rfxn.com/appdocs/CHANGELOG.maldetect

Description
Linux Malware Detect (LMD) is a malware scanner for Linux released under the GNU GPLv2 license, that is designed around the threats faced in shared hosted environments. It uses threat data from network edge intrusion detection systems to extract malware that is actively being used in attacks and generates signatures for detection. In addition, threat data is also derived from user submissions with the LMD checkout feature and from malware community resources. The signatures that LMD uses are MD5 file hashes and HEX pattern matches, they are also easily exported to any number of detection tools such as ClamAV.

The driving force behind LMD is that there is currently limited availability of open source/restriction free tools for Linux systems that focus on malware detection and more important that get it right. Many of the AV products that perform malware detection on Linux have a very poor track record of detecting threats, especially those targeted at shared hosted environments.

The threat landscape in shared hosted environments is unique from that of the standard AV products detection suite in that they are detecting primarily OS level trojans, rootkits and traditional file-infecting viruses but missing the ever increasing variety of malware on the user account level which serves as an attack platform.

The commercial products available for malware detection and remediation in multi-user shared environments remains abysmal. An analysis of 8,883 malware hashes, detected by LMD 1.5, against 30 commercial anti-virus and malware products paints a picture of how poorly commercial solutions perform.

DETECTED KNOWN MALWARE: 1951
% AV DETECT (AVG): 58
% AV DETECT (LOW): 10
% AV DETECT (HIGH): 100
UNKNOWN MALWARE: 6931

Using the Team Cymru malware hash registry, we can see that of the 8,883 malware hashes shipping with LMD 1.5, there was 6,931 or 78% of threats that went undetected by 30 commercial anti-virus and malware products. The 1,951 threats that were detected had an average detection rate of 58% with a low and high detection rate of 10% and 100% respectively. There could not be a clearer statement to the need for an open and community driven malware remediation project that focuses on the threat landscape of multi-user shared environments.

Features:
– MD5 file hash detection for quick threat identification
– HEX based pattern matching for identifying threat variants
– statistical analysis component for detection of obfuscated threats (e.g: base64)
– integrated detection of ClamAV to use as scanner engine for improved performance
– integrated signature update feature with -u|–update
– integrated version update feature with -d|–update-ver
– scan-recent option to scan only files that have been added/changed in X days
– scan-all option for full path based scanning
– checkout option to upload suspected malware to rfxn.com for review / hashing
– full reporting system to view current and previous scan results
– quarantine queue that stores threats in a safe fashion with no permissions
– quarantine batching option to quarantine the results of a current or past scans
– quarantine restore option to restore files to original path, owner and perms
– quarantine suspend account option to Cpanel suspend or shell revoke users
– cleaner rules to attempt removal of malware injected strings
– cleaner batching option to attempt cleaning of previous scan reports
– cleaner rules to remove base64 and gzinflate(base64 injected malware
– daily cron based scanning of all changes in last 24h in user homedirs
– daily cron script compatible with stock RH style systems, Cpanel & Ensim
– kernel based inotify real time file scanning of created/modified/moved files
– kernel inotify monitor that can take path data from STDIN or FILE
– kernel inotify monitor convenience feature to monitor system users
– kernel inotify monitor can be restricted to a configurable user html root
– kernel inotify monitor with dynamic sysctl limits for optimal performance
– kernel inotify alerting through daily and/or optional weekly reports
– e-mail alert reporting after every scan execution (manual & daily)
– path, extension and signature based ignore options
– background scanner option for unattended scan operations
– verbose logging & output of all actions

Source Data:
The defining difference with LMD is that it doesn’t just detect malware based on signatures/hashes that someone else generated but rather it is an encompassing project that actively tracks in the wild threats and generates signatures based on those real world threats that are currently circulating.

There are four main sources for malware data that is used to generate LMD signatures:
Network Edge IPS: Through networks managed as part of my day-to-day job, primarily web hosting related, our web servers receive a large amount of daily abuse events, all of which is logged by our network edge IPS. The IPS events are processed to extract malware url’s, decode POST payload and base64/gzip encoded abuse data and ultimately that malware is retrieved, reviewed, classified and then signatures generated as appropriate. The vast majority of LMD signatures have been derived from IPS extracted data.
Community Data: Data is aggregated from multiple community malware websites such as clean-mx and malwaredomainlist then processed to retrieve new malware, review, classify and then generate signatures.
ClamAV: The HEX & MD5 detection signatures from ClamAV are monitored for relevant updates that apply to the target user group of LMD and added to the project as appropriate. To date there has been roughly 400 signatures ported from ClamAV while the LMD project has contributed back to ClamAV by submitting over 1,100 signatures and continues to do so on an ongoing basis.
User Submission: LMD has a checkout feature that allows users to submit suspected malware for review, this has grown into a very popular feature and generates on average about 30-50 submissions per week.

Signature Updates:
The LMD signature are updated typically once per day or more frequently depending on incoming threat data from the LMD checkout feature, IPS malware extraction and other sources. The updating of signatures in LMD installations is performed daily through the default cron.daily script with the –update option, which can be run manually at any time.

An RSS feed is available for tracking malware threat updates: http://www.rfxn.com/api/lmd

Detected Threats:
LMD 1.5 has a total of 10,822 (8,908 MD5 / 1,914) signatures, before any updates. The top 60 threats by prevalence detected by LMD are as follows:

base64.inject.unclassed     perl.ircbot.xscan
bin.dccserv.irsexxy         perl.mailer.yellsoft
bin.fakeproc.Xnuxer         perl.shell.cbLorD
bin.ircbot.nbot             perl.shell.cgitelnet
bin.ircbot.php3             php.cmdshell.c100
bin.ircbot.unclassed        php.cmdshell.c99
bin.pktflood.ABC123         php.cmdshell.cih
bin.pktflood.osf            php.cmdshell.egyspider
bin.trojan.linuxsmalli      php.cmdshell.fx29
c.ircbot.tsunami            php.cmdshell.ItsmYarD
exp.linux.rstb              php.cmdshell.Ketemu
exp.linux.unclassed         php.cmdshell.N3tshell
exp.setuid0.unclassed       php.cmdshell.r57
gzbase64.inject             php.cmdshell.unclassed
html.phishing.auc61         php.defash.buno
html.phishing.hsbc          php.exe.globals
perl.connback.DataCha0s     php.include.remote
perl.connback.N2            php.ircbot.InsideTeam
perl.cpanel.cpwrap          php.ircbot.lolwut
perl.ircbot.atrixteam       php.ircbot.sniper
perl.ircbot.bRuNo           php.ircbot.vj_denie
perl.ircbot.Clx             php.mailer.10hack
perl.ircbot.devil           php.mailer.bombam
perl.ircbot.fx29            php.mailer.PostMan
perl.ircbot.magnum          php.phishing.AliKay
perl.ircbot.oldwolf         php.phishing.mrbrain
perl.ircbot.putr4XtReme     php.phishing.ReZulT
perl.ircbot.rafflesia       php.pktflood.oey
perl.ircbot.UberCracker     php.shell.rc99
perl.ircbot.xdh             php.shell.shellcomm

Real-Time Monitoring:
The inotify monitoring feature is designed to monitor paths/users in real-time for file creation/modify/move operations. This option requires a kernel that supports inotify_watch (CONFIG_INOTIFY) which is found in kernels 2.6.13+ and CentOS/RHEL 5 by default. If you are running CentOS 4 you should consider an inbox upgrade with:
http://www.rfxn.com/upgrade-centos-4-8-to-5-3/

There are three modes that the monitor can be executed with and they relate to what will be monitored, they are USERS|PATHS|FILES.

       e.g: maldet --monitor users
       e.g: maldet --monitor /root/monitor_paths
       e.g: maldet --monitor /home/mike,/home/ashton

The options break down as follows:
USERS: The users option will take the homedirs of all system users that are above inotify_minuid and monitor them. If inotify_webdir is set then the users webdir, if it exists, will only be monitored.
PATHS: A comma spaced list of paths to monitor
FILE: A line spaced file list of paths to monitor

Once you start maldet in monitor mode, it will preprocess the paths based on the option specified followed by starting the inotify process. The starting of the inotify process can be a time consuming task as it needs to setup a monitor hook for every file under the monitored paths. Although the startup process can impact the load temporarily, once the process has started it maintains all of
its resources inside kernel memory and has a very small userspace footprint in memory or cpu usage.

See http://www.rfxn.com/appdocs/README.maldetect for more details on inotify monitoring.

Funding:
Funding for the continued development and research into this and other projects, is solely dependent on public contributions and donations. If this is your first time using this software we ask that you evaluate it and consider a small donation; for those who frequent and are continued users of this and other projects we also ask that you make an occasional small donation to help ensure the future of our public projects.

211 Replies to “Linux Malware Detect”

  1. I’ve not been able to get maldet to clean anything. It ALWAY resets the base64 file, even if I chattr +i the thing.

    Also not quarantining anything. Creates .info files, but infected files remain in users folder.

    Totally useless other then as a reporting tool

    1. This is certainly not the case for most users and sounds like an issue in your setup. If you detail a bit about your deployment perhaps we can pinpoint the issue.

  2. Hi all,

    Today (march 7 2012) I downloaded LMD 1.4.1 with wget http://www.rfxn.com/downloads/maldetect-current.tar.gz to 2 systems (1 local and 1 hosted server).

    On the remote system I immediately did a scan and it found that gzbase64.inject.unclassed was infected, which was put in quarantine

    Now I am quite a newbe and maybe I did something wrong so I dont want to create a panic, but this file was found in maldetect-1.4.1/files/clean/gzbase64.inject.unclassed

    Therefore to me it looks like that this file was included in the LMD download because i did not download or install anything else

    Is gzbase64.inject.unclassed indeed infected with malware as it was detected by LMD itself as such?

    If so what is my situation? am I infected now, without nowing, because the installation of MLD is done with sudo right?

    Another strange thing is that my local installation went without errors, but when I for instance do sudo maldet -u I get ‘command not found”, also when I do it it the maldetect dir. Maldet does not run at all!

    So am I doing something wrong or is LMD infected?

    1. Hi, Jan. I’ve just installed LMD for the first time, but since no one else has responded to your comment, I thought I would. Just keep in mind I’m not an experienced LMD user.

      If you look at the contents of the file in question, you’ll see it’s actually a sed command that uses regular expressions to find and clean certain generic ways of injecting hidden malware instructions into PHP files. Because it contains the signature of that type of malware injection, LMD detects it as malware. It isn’t.

      I’m a new user, but so far I have made a point of manually verifying each reported infection. I use LMD to recursively scan user home directories since that’s where the real vulnerability is (if I’m uploading infected software or not securing my server in general, LMD is probably too little and too late!).

      I personally would not think it wise to include the LMD directory in LMD’s malware scans, so, yes, I think you’re doing something wrong. If malware were to infect LMD itself, odds are it would alter LMD not to detect it, so there’s not much point in this. I generally think it’s not smart to try to detect and purge infections using an infected tool or framework.

      See also section 13 of the README file distributed with LMD. It’s always a good idea to “RTFM” before posting questions publicly. πŸ™‚

      While I would encourage you to post any response you might have as a reply here, I honestly may not see it. If you want to be sure of getting my attention, you can also connect with me on Twitter. My Twitter handle is @sustainablylush.

  3. Hi Ryan:

    For the next update, version, etc., can you put a cap and some form of clean up for /usr/local/etc/maldetect.bk* files?

    Those directories and files can quickly fill up a disk if not manually managed (it is not a huge amount of work — typically checking once every four to eight weeks, but still… automation is nice).

    Thank you!

  4. Hello!

    Thanks a lot for the wonderful script, we are now using it to scan out customers’ websites for malware each night.

    The only thing which we would really like to have is a nightly report via email (regardless if malware is found or not).
    When we receive an email, we know that lmd is working fine.

    Else we would have to check every day manually in the logs if lmd has run…

    That would be a _huge_ timesaver, because the worst thing is when you THINK that your system gets scanned but for some reason it doesn’t πŸ˜‰

    Thanks a lot and best regards,

    Chris

    1. I will look into adding an option to enable this however by default LMD only alerts when it has found something, as for example if you had hundreds of servers it would be very noisy to receive empty emails stating nothing more than LMD found no malware.

  5. I’ve been using LMD for some time and love it, with one minor issue (an annoyance really). When you run “maldet -d” to force an update at the command line, if there is a new version of the application available, it downloads the new version, and reinstalls it in /usr/local. If you have maldet configured to run from an alternate location (for example, setting inspath=/opt/maldet in the main maldet script), the update attempts to put a new install on the system instead of updating the existing installation (due to “inspath” being hardcoded in install.sh).

    Passing an installation path to install.sh then updating the maldet script with this new path would be an ideal solution.

    Looking forward to your thoughts on this and thanks for a great scanning solution.

    JN

  6. Hi ryan. I used your cool Maldetect a week ago, and it saved my hosting reputation. So I donated my $10.
    I decided to run it on a daily basis from crontab, so I put on purpose an infected file Maldetect found a week ago on a specific folder, just to be sure that it runs fine. Guess what, it doesn’t find it anymore!
    Very strange. Also, I started to get a strange “find” error, as detailed below:

    [email protected] [~]# /usr/local/maldetect/maldet -a /home/sch
    Linux Malware Detect v1.4.1
    (C) 2002-2011, R-fx Networks
    (C) 2011, Ryan MacDonald
    inotifywait (C) 2007, Rohan McGovern
    This program may be freely redistributed under the terms of the GNU GPL v2

    maldet(9491): {scan} signatures loaded: 8873 (7009 MD5 / 1864 HEX)
    maldet(9491): {scan} building file list for /home/sch, this might take awhile…
    /usr/bin/find: paths must precede expression
    Usage: /usr/bin/find [path…] [expression]
    maldet(9491): {scan} scan returned zero results, please provide a new path.

    What that “/usr/bin/find: paths must precede expression” would be?

    Thanks for your good work

    1. LMD scans for file changes/modifications in the last 24hr so if you simply moved the file into the path it’s modification/creation date is likely still reflective of when it was originally created.

      Please run the following:
      sh -x /usr/local/maldetect/maldet -a /home/sch 2> /root/maldet.debug

      Then send me the an email to ryan at rfxn.com with the attached maldet.debug file.

    1. Peter,
      You can place the file names or the paths to the directories in question into the /usr/local/maldetect/ignore_paths file to prevent these files from being scanned. Or, you can place the base64.inject.unclassed string into ignore_sigs to ignore the entire rule class.

  7. I see there are a number of directories with the names maldetect.bk12345 in /usr/local/ (where 12345 are various numbers), are these archives that can safely be removed?

  8. Hi Ryan,

    Thanks for the hard work and making such great software, we’ve just sent over a donation to show our appreciation.

    Quick question:

    On some of our customer servers we’ve customized /etc/cron.daily/maldet to scan in specific paths not included in the defaults you provide.

    It looks like when the software is automatically upgraded the changes we make are wiped out in /etc/cron.daily/maldet and it no longer scans the correct directories.

    Can we chattr +i /etc/cron.daily/maldet so it is not modified in the future, or will that potentially break new updates? Any other options?

    Thanks in advance.

    -PJF

    1. Setting the cronjob chattr +i will not break anything. That said, I will work on the update function to have it detect when the cronjob has been modified and preserve it across upgrades.

  9. Ryan, On the inotify monitor process, I have yet to see it stay up on a server Seems like after so long or a few days it dies out. What is the cause of this and how to prevent it?

    You can catch me on aim – procrace and I can show you some servers I have seen this happen on

  10. Need Help Please – Web hosting account suspended due to false positives from seo.classes.php

    I believe my web host supplier is runing your program and after loading up an oscommerce addon to my site my hosting account is automatically suspended due to a false positive from the seo.classes.php file?

    I’ve taken it up with the host two days in a row now but they run a low cost model and aren’t big on customer service that requires any manual adjustments. I appreciate the fact that they probably have thousands of clients on their servers, but surely there is a way they can easily allow this file to not be automatically quarantined and the account suspended. It doesn’t help my online rep.

    I would also suggest that the “Account Suspended” text be changed to something like “Site Offline”. They are both technically correct but the latter is substantially less damaging.

    Any advice would be appreciated!

    1. LMD provides a number of facilities for ignoring false positives including an ignore_paths file which a full path to the false positive file can be placed in or an ignore_sigs file where problematic signatures can be ignored entirely. I would recommend you advise your host to place the full path to your file into /usr/local/maldetect/ignore_paths and it should take care of the issue. What host are you using if I may ask?

  11. Thanks for this excellent product!

    I found a threat that wasn’t detected by maldet.
    I tried to upload it with the -c option but I get:

    550 Can’t change directory to incoming: Permission denied

    Any ideas?
    Thanks!

  12. Hi Ryan,

    thanks for working hard on the script πŸ™‚

    I’m on Debian Squeeze (64bit) and am still encountering the same problem as the poster of comment #106, maldet reports that says that there are no inotify processes found.

    {mon} no inotify process found, check /usr/local/maldetect/inotify/inotify_log for errors.

    The log stays empty and when i manually execute inotifywatch without arguments it seems to run and complains that there are no files specified to watch.

    I’ve manually modified maldet to not grep for /home in /etc/passwd but for /var/www (which is my home dir for webhosting users) and it did detect and set inotify to a more reasonable value (instead of 0), however it still reports there are no inotify processes found.

    Do you have an idea on how to fix that?

Leave a Reply

Your email address will not be published. Required fields are marked *