LMD 1.3.7: Milestones, Fixes & Signature Updates

Today marks the release of LMD 1.3.7, which is a minor release update that fixes a few bugs and is also the final 1.x release before version 2.0 as described in the LMD: one year later blog post. The bug list for LMD has remained very small over the last 6 months and this release reflects that by fixing the current outstanding bugs.

Changes 1.3.6 => 1.3.7:
[Fix] package ownership at some point got set to uid 501 instead of root
[Fix] daily cronjob now checks ps output for inotifywait proc instead of pidof
[Fix] monitor mode users would exit prematurely if a user home path did not exist
[Fix] a file hijacking race condition existed with quarantine mode restore function
[Fix] inotify max_user_instances value was being set to a value that would cause inotifywait
to fail

A thanks goes out to Mark McKinstry of Nexcess.net for assistance tracking down and fixing the issue with inotifywait reporting on some systems that inotify support did not exist in the kernel, when it actually did, this was an issue with the value maldet was setting for inotify max_user_instances. A thanks also goes out to Jeff Patersen from webhostsecurity.com for identifying and bringing the file hijacking race condition to my attention. This issue had the potential, under certain circumstances, to allow a user to gain access to root-owned files in user-readable paths. These fixes on their own are reason enough for all users to update, the -d|–update-ver command switches will take care of all update business for users so there is no reason to not update (i.e: # maldet -d).

Today I have also put up a small set of signature updates on top of the regular daily queue processing, this includes 25 HEX signatures for various items in the review queue as well as associated file hashes. This brings the project to over 5,000 signatures, a milestone that has been a long time coming and one that sets this project apart from all other malware projects in the Linux world. Even the top tier AV vendors and open source project ClamAV lack the depth of malware signatures that LMD brings to the Linux community. At the moment, the project is growing by an average of 14 signatures per day with a review queue that I still need to finish processing of over 1300 user submissions.

We also can celebrate another milestone this month, with passing 3,000 confirmed installations of LMD (3,241 as of this writing). We can determine this by checking the number of unique IP addresses (servers) that check-in daily to the rfxn.com server for signature updates. The total downloads of LMD sit at 12,952 to date, which is roughly where we expect it to be having had 3 major releases (minor releases dont get much attention) that most users would have installed or updated to.

As a holiday gift to all LMD users, I am making it my goal to have all pending items in the review queue processed and signatures created by the end of December, so keep your eyes open and i’ll make a post when that has been completed.

Donations: By The Numbers

I was recently asked by someone about the donations that rfxn.com receives, more specifically what it amounts to. In the interest of answering this person and anyone else who may be curious, I thought I would put together a small post about it.

Firstly, what needs to be said is that although the projects have been active for nearly 8 years and will pass 700,000 downloads sometime in January 2011, I have only been accepting donations since late January of 2006. Around this time, is when rfxn.com had a shift from a for-profit managed services provider which offered our projects as a contribution back to the community, to simply a community site dedicated to the projects as I moved into full-time employment.

In that time, almost 5 years, there has been less than 100 donations to the projects (73 to be exact) or an average of 14 donations per year or little more than 1 per month. The average donation amount is $35 and there are typically only two donations per year totaling more than $100. The yearly donation average is $521 and monthly average is $43. There has only been 3 repeat donors and they donate on average of once every 18 months. The sum of all donations to date since January of 2006 is $2,608 USD, with $73 of that going towards transaction fees.

The context of this is that there are currently 10 maintained projects, comprising 23,706 lines of code, of which the projects receive an average of 8,000 downloads per month. There has been a little over 690k downloads to date and of the projects that access rfxn.com servers regularly we can derive that there is currently at least 24,629 unique IP addresses (servers) running one or more projects (only 3 projects regularly access data on rfxn.com post-install, so that figure is an order of magnitude larger relative to total downloads).

I have recently added a donation roll page that lists every donation to rfxn.com projects to date and a widget on all pages that displays the 5 most recent donations. This is an effort to further acknowledge the minority of individuals that contribute financially to the projects and to be completely transparent about donations. As the above clearly shows, the donations are not a means for survival of the projects let alone a means for me personally to survive, the projects are developed in my spare time between my full-time job and life. This spare-time nature to maintaining the projects has over the years resulted in a fair share of less-than-constructive comments about the projects and frequency of updates, although some of these comments do have merit, in general the projects are mature, stable and have stood the test of time to prove they are relevant just as much today as they were yesterday.

I hope this post has helped to better illustrate the personal commitment I have towards the projects, what (financial) incentives exist surrounding the projects and to provide a clear and transparent account of donations.

LMD: One Year Later

With my move back to Canada behind me and adjusting to some new routines with life, its about time to get back into the mix with the projects. Though things have been slow the last couple of months, it has not stopped me from making sure regular and prompt malware updates are released.

Today, we reflect on the first year of Linux Malware Detect, which was released in a very infantile beta release about a year ago. The project has evolved in allot of ways from its original goals, it has certainly changed in every way for the better. What was originally to be a closed project, relegated to mostly internal work related needs, ended up like most of my projects morphing into a public release. The first release saw the world with less than 200 signatures, no reliable signature update method, manual upgrade options and very flawed scanning and detection methods (v 0.7<). Now, we sit at version 1.3.6, with 4,813 signatures, a scanning method that though still needs some work, is far superior than what was originally in place, a detection routine based on solid md5 hashes and hex signatures. We have cleaner rules that can clean some nasty injected malware, we got a fully functional quarantine and restore system, reporting system, real-time file based monitoring, integrated signature updater and version updater and a vibrant community of users that regularly submit malware for review. Yes, LMD has grown up!

The most grown-up part of LMD has to be how signatures are handled and how the processing of them is almost an entirely automated process now, this was detailed a little more in Signature Updates & Threat Database posted in September. The key part here though, is “almost entirely automated”, everyday that the processing scripts run to bring in new malware, there is always a number of files that cant be processed automatically and these are moved to a manual review queue. With how busy life has been the last couple of months, the review queue has slowly risen to 1,097 files pending review. This queue is at the top of my list for tackling over the next couple of days and weeks, its allot of work to review that much malware but it will get done. Many of the files to review are actual user-submissions so if you did submit something and find its still not detected by LMD, this would be why :).

There is still allot on the to-do list for LMD going forward, with the upcoming release of version 2.0 we will see some changes in how LMD does business. The first and to me the biggest will be optional usage statistics, which will allow users to have LMD report anonymized statistics back to rfxn.com. These statistics will show us which malware hits are found on your servers, which in turn contributes towards better focus on what type of malware threats are prioritized in the daily processing queue for hashing & review. The statistics will also help create informative profiles on the soon-to-be-released dailythreats.org web site about how maldet is used and what are the most prevalent threats in the wild.

Other additions to LMD 2.0 will be a refined scanner that will provide greater speed with large file sets (50k – 1M+ files), an ability to fork scans to the background, better and more predictable logging format for 3rd party processing of LMD log data, redesigned reporting system, full BSD support, ability to create custom signatures from the LMD command line, expanded cleaner rules, wildcard support for exclude paths, a number of security and bug refinements and as always, more signatures.

If you have any feature requests for LMD 2.0, go ahead and post them as a comment and I will make sure they get added to the list. Thank you to everyone who continues to support rfxn.com projects through donations, feedback and by just using & spreading the word about the projects. I look forward to another year of LMD and seeing it become the premier malware detection tool for Linux and all Unix variant OS’s.