Friday, July 29, 2011

Linux Server Monitoring



Linux Server MonitoringThere is a bunch of Linux Server Monitoring software available. The problem is sifting through it all. The first thing to do is identify what it is you want to monitor then you can find what software will work best for your needs. As far as system monitoring goes there is old school and new school. Any combination of tools could be used and there are also a number of ways you can home brew some monitoring solutions. Click for the large version of the Linux Server Monitoring Image.

Workstation Monitoring

Although not exactly server monitoring, workstation monitoring is a good starting point if you want to explore the available options.
Top
The most basic form of monitoring is top. This comes installed with most distributions, live cd’s and servers. To use it just open up a terminal and type the command.  It will auto update by default every 5 seconds but there are some tweaks to make it more useful, my favorite is the < and keys, which change which field it is ordered by. It is very useful to see what is taking up precious CPU or memory resources, the man page is the best resource to learn more.
htopA slightly newer top, with colors and textual bar graphs.
GKrellMIt’s not quite as old as top, but still old school and its still available in many distributions repositories. Its great to have sitting on your desktop and has a whole range of plugins to make it do some pretty cool stuff.
Torsmo / Conky
Torsmo (http://torsmo.sourceforge.net/) and Conky (http://conky.sourceforge.net/) are pretty much the same thing. Conky started out as a fork of Torsmo which is why I’ve included them under the same category. Basically these are little apps that live on your desktop like GKrellM except in text mode. You can display pretty much any information you want, and run external applications including shell scripts.
phpSysInfo
A tool long forgotten about is phpSysInfo (http://phpsysinfo.sourceforge.net/) It requires a webserver such as apache to run, but provides a broad range of information. This tool really has improved over the years becoming more visually appealing, supporting more operating systems and languages. It gives the most basic information out of the box, and to be honest Top gives you more info, but it can now be extended with plugins, meaning you can do whatever you want with it.
rrdtool
Perhaps for the most hardcore users that wish to and some zing to their monitoring rrdtool (http://www.mrtg.org/rrdtool/) provides a great interface to creating graphs from data. Check out their Gallery for some examples of what rrdtool can do. A lot of the other tools in this document also utilize rrdtool.
Cacti
If you’re not as daring as the die-hard rrdtool users Cacti (http://cacti.net) is a great piece of software that provides an interface and simplifies using rrdtool. You can create your own templates or use the pre-built templates that allow you to monitor a variety of things.

Web Server monitoring

I couldn’t get away with talking about Linux Server Monitoring without mentioning Web Server monitoring. If you’ve ran a website before you know how exciting it can be to monitor your server. *Chuckles*, seriously though these tools can make it more fun and worthwhile. These tools may monitor, but some have very different uses from others. I’ll start with the most basic.
Apache Logs
While not exactly server monitoring software, these deserve a mention. You can  monitor a number of things such as access logs:
tail -f /var/log/httpd/access_log
tail -f /var/log/httpd/ssl_access_log
or error logs:
tail -f /var/log/httpd/error_log
tail -f /var/log/httpd/ssl_error_log
If you want to get really snazzy with it you can use a tool like multitail.
apachetop
A great piece of software that simplifies using a pager and opening the log files and puts it into a top like interface. Handy to see real time statistics.
mod_status
Possibly the most complex apache monitoring software available. I’ve honestly never used this before, but after reading this I may just have to give it a try.

Web Analytic Software

These don’t per say monitor but rather give you statistics. They can be useful to identify issues related to traffic spikes.
Apart from Google Analytics My favorite is analogstats, I have not yet got a chance to run PiWik but it looks pretty good too. Evaluate them all and make a decision.

Database Monitoring

MySQLTop
A hard one to find in some distributions repositories. I actually found mysqltop:http://jeremy.zawodny.com/mysql/mytop by mistake one day. Its similar to top and apache top except it monitors mysql databases.
InnoTop (http://sourceforge.net/projects/innotop)
I perceive innotop to be the same as mysqltop except for innodb.
check_postgresWithout mentioning postgres I wouldn’t be completing the database section. check_postgres (http://bucardo.org/wiki/Check_postgres) is a set of scripts to help monitor it. I’ve always found that configuring postgres can be tricky, and since I haven’t tried this one I’m wondering how hard it would be to set up.

Network Monitoring

What good is all this server monitoring if your network is crap non-functioning? A start would be to monitor your network, after all even if your server is up, its no good if your network doesn’t work.
netstat (http://freshmeat.net/projects/netstat/)
Possibly the simplest form of monitoring for the network. Not very functional for more than a workstation listening post monitor unless you were to hack something together. Worth a mention none the less.
etherape
I’m assuming this is pronounce ether ape as in the chimp variety and not eth-rape, since there is an “e” in between. Etherape is a powerful graphic network monitoring tool. Check out the screenshots to see what I’m talking about.http://sourceforge.net/projects/etherape/
iptraf
This one has been around about 10 or so years, but the website hasn’t been updated in 5. Some may think that it looks like Kismet, but I say Kismet looks like iptraf.
mrtg
If you have the Multi Router Traffic Grapher open to the public it gives them an idea on how much traffic you get. Oh what I’d give to get a sneak at Youtubes graph. Used by script kiddies everywhere to see if their DDoS’s are working. Mrtg makes nice rrdtool graphs and wraps an interface around them.
netmonitor (http://netmonitor.sourceforge.net/)
Top like network interface to view network bandwidth / usage. Updated slightly more recently than iptraf.
Use:
netmonitor –config
to generate a config file, startup netmonitor and watch magic in the making.
jnettop
Project page is here: http://jnettop.kubs.info/wiki/ but the freshmeat pagehttp://freshmeat.net/projects/jnettop/ has working screenshots. This is a top like interface that you guessed it, displays information like top.
ntop (http://www.ntop.org)
I know what your thinking, another top interface, give it a rest already. Well, you are wrong. Although ntop shares its name the user interface is far from the same.
even runs on win32 since it uses the libpcap library.
Smokeping
Sounding like a deep purple song we have smokeping, measure and track your network latency in style. http://oss.oetiker.ch/smokeping/

Linux Enterprise Server Monitoring

Lets get to the part that every one wants to hear about. Linux Server Monitoring forsuits enterprise. So, these are the top picks that are either open source, free or little support cost. I’ll be honest here, I don’t have that much experience in this arena.
Monit
Monit (http://mmonit.com/monit/) was suggested to me by NOGREP while writing this article. I’m not sure how well it scales but it certainly has all the makings of an enterprise solution. It can monitor process, files, and network stuff either locally or on remote hosts. It also has its own web server for graphical server monitoring.
Nagios
The world famous Nagios (http://nagios.org) Huge community here and for good reason. Possibly one of the most robust monitoring software solutions out there. I’ve talked to a few IT managers that swear by it, Nagios is also available in many distributions repositories already making it a great choice. Monitoring, Alerting, Response, Reporting, Maintenance and Planning are the larger areas that Nagios supports.
Zenoss
Perhaps known more for its enterprise services zenoss has a community edition. Look for the community edition link on their home page, at the time of this writing the URL is http://community.zenoss.org. Zenoss provides availability monitoring, performance monitoring, event monitoring, alerting and more. A neat feature is the XML-RPC and REST api’s making it integrable and extensible.  The community edition released under the GPL license.
OpenNMS
OpenNMS Either Network Management software or solution (http://www.opennms.org/) is perhaps geared more towards the network infrastructure side of the house, although as stated before this can be useful. Its perhaps the oldest available and can be highly customized like the others.
PandoraFMS
The Pandora Flexible monitoring system (http://pandorafms.org/) doesn’t mean its bendy. Its a really pretty monitoring system with some unique features I haven’t seen in any of the others. The web cam overlay is nice, although I don’t really know how practical. The graphs are pretty and not so rrd like and the maps look awesome. The network auto generation is not unique but gives a cool visualization of a network fairly quickly.

Conclusion

This could be the most comprehensive list I’ve ever come up with. As you can see there is a ton of open source Linux Server Monitoring software available. Give them a try, use the comments to tell what you think of if any nuggets were missed.