Process Resource Monitor

Current Release:
http://www.rfxn.com/downloads/prm-current.tar.gz
http://www.rfxn.com/appdocs/README.prm
http://www.rfxn.com/appdocs/CHANGELOG.prm

Description:
Process Resource Monitor (PRM) is a CPU, Memory, Processes & Run (Elapsed) Time resource monitor for Linux & BSD. The flexibility of PRM is achieved through global scoped resource limits or rule-based per-process / per-user limits. A great deal of control of PRM is maintained through a number of ignore options, ability to configure soft/hard kill triggers, wait/recheck timings and to send kill signals to parent/children process trees. Additionally, the status output is very verbose by default with support for sending log data through syslog.

There is no shortage of usage methods for PRM, especially when leveraging the ignore and kill options with the rule based system. You can easily take control of those run-away aspell/pspell processes eating 80% of memory, errant exim processes threading off 500 processes or prevent scripts executed under apache from running for hours at a time.

Features:
– global resource limits
– per-process/per-user rule based resource limits
– rules only run mode
– alert only run mode
– ignore root processes option
– process list global ignore file
– user based global ignore file
– command based global ignore file
– regex based global and/or per-rule ignore variable
– global scoped resource limits
– per-process or per user rule based resource limits
– set custom kill signals (i.e: SIGHUP)
– parent process kill option to terminate an entire process tree
– kill trigger/wait times allow rechecking usage over a period of time
– kill restart option to execute a custom restart command
– all kill/resource/ignore options can be global or rule defined
– easy to configure percentage of total CPU & MEM limits
– total number of processes limits
– elapsed run time limits

Funding:
Funding for the continued development and research into this and other projects, is solely dependent on public contributions and donations. If this is your first time using this software we ask that you evaluate it and consider a small donation; for those who frequent and are continued users of this and other projects we also ask that you make an occasional small donation to help ensure the future of our public projects.

47 Replies to “Process Resource Monitor”

  1. Hi Ryan,

    PRM currently assumes the logger command is found under “/bin/logger”, but this is not always the case, it would be very helpful if the script used the which command to find the location of the logger command before using it instead. Then it wouldn’t matter where it lived.

    If it doesn’t find the the logger command under the /bin dir then it just writes to the messages log directly which is fine unless your running a multi server setup where syslog is forwarding all messages to a central logging server, because these messages will be missed off from the logs.

      1. In v1.1.2 the “shell command paths now auto discovered” causes prm to try running /bin/$LOGGER where $LOGGER is the result of `which logger` this means if $LOGGER = /usr/bin/logger then it tries to run /bin//usr/bin/logger which won’t exist. I don’t believe the `/bin/` part before $LOGGER is needed (the if statement above checking if the file exists also only checks the $LOGGER path without the `/bin/`).

  2. The system works but I get, when a process is killed:
    prm[25271]: checking proc pid:5066 {user:root cmd:yum-updatesd} ./prm: line 376: [: : integer expression expected

  3. Hi Ryan and All!

    Trying PRM for the first time, but already excited by the possibilities it is supposed to provide. I am trying to limit CPU and RAM consumption per virtual server (account, domain, website, you name it) on my dedicated server and hope PRM can be of use in my situation.

    The last versions descriptions says “- per-process/per-user rule based resource limits”, I would love to see little bit detailed elaboration on how exactly to limit CPU and RAM resources per users, but it would be even more fantastic if PRM would allow limit resources for the whole group of users.

    If it is not possible to limit the user groups, will it be possible in the future releases of PRM?

  4. I love prm so far however it doesn’t seem to kill processes running in mono, is there anything I can tweak to get it to kill the following: (The cpu was at 199.4% and it wasn’t being killed)

    735892 1000 15 0 519m 183m 9.9m S 199.4 0.6 1154:25 mono program.exe

  5. Hi Ryan,

    First of all, thanks for your scripts. Really great work!

    I discovered some problems.
    1) I want to monitor processes named php-cgi and created the file:
    /usr/local/prm/rules/php-cgi.cmd
    But the config file being read in that case was:
    /usr/local/prm/rules/phpcgi.cmd

    I had to change the line (in /usr/local/sbin/prm):
    . $RULES/$include_cmd.cmd
    to:
    . $RULES/$cmd.cmd
    in order to make it work.

    Maybe you can fix a better solution, or give an explanation of $include_cmd and how to name .cmd files with hypen in the command name?

    2) Also when killing of an old version of prm that has hung the prm pid didn’t get updated.

    I had to add the line:
    echo “$$” > $inspath/tmp/prm.pid

    After:
    if [ -f “$inspath/tmp/prm.pid” ]; then
    opid=`cat $inspath/tmp/prm.pid`
    kill -9 $opid >> /dev/null 2>&1
    eout “cleared stale lock file ($DIFF > $LOCK_TIMEOUT) and killed pid $opid.”
    else
    eout “cleared stale lock file ($DIFF > $LOCK_TIMEOUT).”
    fi

    Maybe you can add some comments.

    Thanks

  6. Me again! I’ve realized it is a global limit.

    How can I add a limit to ALL users (so no exception rules will be needed) saying that they can’t use more than X% CPU and Y% Memory each?

  7. Hi there.

    I’ve installed this (PRM) on my servers. However I want to limit users to X% CPU and Y% Memory. I’m slightly confused about the following lines:

    MAX_CPU=”X”
    MAX_MEM=”Y”

    Do they limit the global usage (so all accounts together can’t go over X and Y) or are they for a per user basis? (so individual accounts can’t go over X and Y)

    Does this make sense?

    As soon as I’ve figured out this plugin properly (I’m a bit of a SSH and server newbie) and see whether it fits my needs or not I’ll be very happy to donate 🙂

  8. Oh, sorry, one more…

    When I install in a WHM/cPanel server, do I install in the /root directory?

    If I don’t install in /root, where would I install it?

    Thanks!


  9. Peter M. Abraham:

    Ryan, given
    CMD: /usr/bin/perl mail.cgi
    What file name would I have to use for the command?
    Could mail.cgi.cmd or would I have to use perl.cmd ?
    The former (mail.cgi focus) would be better.
    Thank you.

    Great utility, really enjoying so far, but I would be curious to know about the situation above as well or what the rule file names would be for cron jobs like:

    /path/to/the/file.cgi
    or
    /usr/bin/php -q /path/to/the/file.php
    or
    /usr/nice /usr/bin/php /path/to/the/file.php

    Or is there a better way to monitor cgi and php scripts?

    Thanks!

  10. Ryan, given

    CMD: /usr/bin/perl mail.cgi

    What file name would I have to use for the command?

    Could mail.cgi.cmd or would I have to use perl.cmd ?

    The former (mail.cgi focus) would be better.

    Thank you.

  11. Hi Ryan:


    Ryan M.:

    I have made some changes to try correct this issue, if you can go ahead and run a clean install of PRM you should find the error fixed. Please let me know how it goes, thanks.


    Peter M. Abraham:

    Hi Ryan:
    To see the error, change the rules only from 1 (default) to 0 on a fresh installation.
    Thank you.

    FIXED. Thank you!!!!

  12. I have made some changes to try correct this issue, if you can go ahead and run a clean install of PRM you should find the error fixed. Please let me know how it goes, thanks.


    Peter M. Abraham:

    Hi Ryan:

    To see the error, change the rules only from 1 (default) to 0 on a fresh installation.

    Thank you.

  13. Hi Ryan:

    I removed /usr/local/prm, re-installed (now no default rules, rules directory is empty), and I still get “/usr/local/sbin/prm: line 230: [: : integer expression expected” when I run prm -s

    Thank you.

  14. Peter,
    It sounds to me like you have installed PRM over an old version and it imported the old config and rule files into the new release which are not compatible. PRM 1.x comes with NO default rules, the valid naming convention for rules are USERNAME.user (i.e: nobody.user) and COMMAND.user (i.e: exim.user). Please refer to the README file for some detailed examples of rules.

    I think the integer error you are receiving may also be due to the config and rule files imported from pre-1.x version. Go ahead and do an rm -rf /usr/local/prm* and fresh install, let me know if you continue to experience issues, thanks.


    Peter M. Abraham:

    Ryan, in the readme file, you have the naming convention of USERNAME.user and COMMAND.cmd.

    Yet, in the default rules, there is “proftpd” (without an extension), httpd (without an extension), etc.

    Is that intentional or should they be renamed httpd.cmd, proftpd.cmd and so on?

    Thank you.


    Peter M. Abraham:

    Hi Ryan:

    Also, on the latest version (1.07) on CentOS 5.5 64-bit (latest kernel), when I run prm -s I receive the following errors:

    /usr/local/sbin/prm: line 230: [: : integer expression expected

    The above goes on for some time.

    Thank you.

  15. Hi Ryan:

    Also, on the latest version (1.07) on CentOS 5.5 64-bit (latest kernel), when I run prm -s I receive the following errors:

    /usr/local/sbin/prm: line 230: [: : integer expression expected

    The above goes on for some time.

    Thank you.

  16. Ryan, in the readme file, you have the naming convention of USERNAME.user and COMMAND.cmd.

    Yet, in the default rules, there is “proftpd” (without an extension), httpd (without an extension), etc.

    Is that intentional or should they be renamed httpd.cmd, proftpd.cmd and so on?

    Thank you.

  17. I have made some changes to PRM and put up version 1.0.7, please give it a spin and see if that works for you, thanks.


    Ali:

    I have FreeBsd Ver.7 , the legacy versions of PRM was working fine.(may be 2003-2004)
    But after upgrade to latest one , the first time of
    prm -s
    Exits with error
    —could not find needed binary /usr/bin/exprt, aborting.
    what must I do?
    What is the “exprt”?

  18. I have FreeBsd Ver.7 , the legacy versions of PRM was working fine.(may be 2003-2004)
    But after upgrade to latest one , the first time of
    prm -s
    Exits with error
    —could not find needed binary /usr/bin/exprt, aborting.
    what must I do?
    What is the “exprt”?

  19. The file: http://www.rfxn.com/downloads/prm-current.tar.gz should extract as prm-1.0.6. What version are you seeing when you download? Also make sure you have no older copies of prm-current.tar.gz in the folder you are downloading too an accidentally extracting it instead.

    A direct link to the latest version is at:
    http://www.rfxn.com/downloads/prm-1.0.6.tar.gz


    Chuck:

    Hi Ryan,

    I’m a little confused at where to get the new tarball. If I click current I still get the old version. Can’t wait to try the new script.

    Chuck

  20. Hi Ryan,

    I’m a little confused at where to get the new tarball. If I click current I still get the old version. Can’t wait to try the new script.

    Chuck

  21. Yup, exactly that, please be sure to check the README file for some usage examples under section 4.


    Son Nguyen:

    Thanks for all your works Ryan! For the new PRM, is upgrading is just as simple as install the new package and it will overwrite the old files?

  22. If prm doesnt use, $6, then just replacing cmd with comm will fix the issue:

    [[email protected] prm]# ps -A -o “user pid pcpu pmem comm” | grep bash
    root 12654 0.0 0.2 bash


    Vincent:

    Hi Ryan,
    In my server the ps -A command returns the following line:
    root 8848 0.0 0.2 -bash
    making $5 “-bash” and returning the error:
    ====
    Usage: grep [OPTION]… PATTERN [FILE]…
    Try `grep –help’ for more information.
    ====
    I did a quick fix by changing line 134 from:
    cmd=”$5″
    to
    cmd=”$(echo $5 | sed ’s/\b\-//’)”
    Hope this helps improve your script.
    Thanks much.
    Vincent

  23. Hi Ryan,

    In my server the ps -A command returns the following line:
    root 8848 0.0 0.2 -bash
    making $5 “-bash” and returning the error:
    ====
    Usage: grep [OPTION]… PATTERN [FILE]…
    Try `grep –help’ for more information.
    ====
    I did a quick fix by changing line 134 from:
    cmd=”$5″
    to
    cmd=”$(echo $5 | sed ‘s/\b\-//’)”

    Hope this helps improve your script.

    Thanks much.
    Vincent

  24. At the moment, PRM has no alert-only feature but I am in the process of rewritting PRM as we speak and will add it to the feature list. It should be out sometime in the next day or so, as it not really a complex project to rework.

  25. If I prefer to limit resources for some users and monitor only resources for other is this possible and would it fall under rules rather than the prm.conf or would changes be required for both.

    what I was thinking of is the exim rule in general. on most accounts I would only want to monitor, then if I found misuse would prefer to kill as well but only on abusive accounts.

    TIA

  26. When PRM finds a process that has exceeded resources, it will wait the amount of time defined by WAIT=”” before rechecking the process and finally kill the process after KILL_TRIG=”” number of times.

    So the default config tells PRM to recheck a process 3 times with 12 seconds between each check

    The LC value simply tells PRM which load value to look at, Linux has a 1, 5 and 15 minute load average and that is represented by LC with 1,2,3 values for the respective load value.

    I know PRM is a little counter intuitive on some of it options and it is due for a complete rewrite but it does work and at the end of the day thats what counts.

  27. Does WAIT=”12″ depend on cronjob? What does LC used for (do I need to run cronjob every 5 minutes even I have set LC=”1″)?

Leave a Reply

Your email address will not be published. Required fields are marked *