http://www.infoworld.com/slideshow/106686/12-clever-cloud-tools-devs-and-ops-221000?source=fssr#slide1
Look to these automation, orchestration, and configuration management tools to keep your server farms running smoothly..
Clever cloud tools
Was it only a few years ago when deploying a single server was a big occasion? First we needed to unpack it, mount it, test it, and install the software. Then we could put it to work.
Today, we're asked to spin up 200 servers before lunch, configure them, crunch some numbers, and return them before the 1 p.m. staff meeting. Time is money, and servers are bought by the hour now.
That explains the explosion of interest in tools that let us manage hundreds of machines with a few quick clicks. Here's a list of some of the most intriguing new tools for making our server farms jump to attention in less time than it took to slice open the tape on a server box.
Ganglia
More than a dozen years ago, developers at U.C. Berkeley looked at the clusters of computers throughout the campus and realized they needed a way to keep track of them -- so they built and rebuilt Ganglia. Today, Ganglia is one of the most comprehensive sets of tools for monitoring a diverse cluster of machines. It collects statistics with a lightweight protocol and graphs them so that you can watch performance over time.
Distributed with the BSD license.
Graphite
There are hundreds of graphing libraries and visualization tools, but they're usually aimed at a fairly static collection of data. Graphite is designed to support real-time data from dozens of streams. The data goes in one side of the application, where it's stored in a real-time, scalable database optimized to handle the numerical streams of information. The system is designed to keep up with fast streams of data from multiple sources. There's even an elaborate caching system to make sure the graphs are up to date.
The back end is written in pure Python, and the Web app on top of Django. The graphing is done with the Cairo libraries. Orbitz built the software to handle internal monitoring, then released it with the Apache...
Tattle
Tools like Graphite and Ganglia may make it easier to monitor the state of the system, but they require someone to sit and watch the screen as the lines dance up and down. Tattle is designed to watch the graphs for you, then send off alerts if a line crosses over into some place it's not supposed to be. These alerts usually go out as text messages and emails, but you can configure them with a plug-in if you want special behavior like a warning siren and flashing red light connected to an Arduino controller.
Written in PHP for both Graphite and Ganglia.
Logstash
There are only three things that are certain for sys admins: death, taxes, and logs full of events. The earth goes 'round the sun, and the log files fill up with entries about visits to the websites and all of the random housekeeping around the stack.
Logstash is an open source package designed to put a pretty face on the endless streams of data. It parses the entries, segments them, and builds graphs so that you can drill down looking for items that need investigating, all through the Web interface. If you need to customize the system, you can write new filter plug-ins that fit into the flow of data.
Distributed under the Apache 2.0 license.
Kibana
Logstash may build elegant, rolling log files, but these can only be searched with basic queries. Kibana is glue between Logstash and Elasticsearch. It dumps Logstash log information into Elasticsearch so that you can write Lucene queries to find what you want. Not only can you use Boolean searches and wild cards to pick through the log files, but Kibana goes one step further by running these queries periodically and packaging the results into a dashboard that's also published with RSS and some other standards.
Available under the MIT license.
CloudVelocity
Everyone loves the idea of the cloud, but no one wants to let go of the security of owning the servers. CloudVelocitywants to ease companies into the cloud (specifically Amazon Web Services for now) by offering a hybrid that mixes the cloud machines with the private boxes. Moving data and operations between the two is meant to be as easy as possible, perhaps as easy as dragging some icons across a Web page. The company promises to let you test your operations with separate test rigs built by cloning existing machines. Now if only it would let us clone ourselves so we can get everything done that the management wants.
Ravello Systems
Another tool for easy cloud migration, Ravello is a layer that sits on top of the major clouds (Amazon, HP, Rackspace) and helps development teams deploy multitier apps into these clouds for testing. You upload the VMs into Ravello and use the drag-and-drop GUI to map out their relationships; Ravello spins them up in the cloud while preserving their network and storage configurations. It's one key feature lets you get several VMs running on just one cloud machine, a nice enhancement if you happen to have a bunch of lightweight VMs that don't need a full machine.
Vagrant
A long time ago, developing software was more straightforward. You had your favorite editor and the compiler. Your file either compiled or it didn't. Today, life is much more complex. There are libraries galore and extra tools for pre-processing or post-processing or housekeeping. All of these are updated periodically, ensuring that everyone on the development team ends up with diverging environments. Eventually, code will compile on some machines and not others, causing the developers to start pointing fingers.
Vagrant juggles all of the packages for all of the tools so that everyone is working with the same environment. It smooths out the differences and ensures that the libraries and dependencies are in sync, saving everyone the hair-pulling and blame-casting.
Available under the MIT license.
SaltStack
Keeping a cloud of machines working together is not so simple, especially if you want new machines to come and go on demand. Salt is an open source cloud management tool that offers a Python-based communications backbone that links together the "master" and "minion" nodes. The master issues orders, and the minions carry them out. SaltStack, a "branch" of Salt backed by professional support, boasts deployments of several hundred thousand managed machines.
Salt is developed under the Apache 2.0 license.
Ubuntu Juju
Ubuntu's Juju is a project that starts to hide the machines behind a wall of services. In Juju the façade is called a "charm," which is the packaging wrapped around the service. Behind the façade is a load balancer that can ask any number of machines to answer the request for data. If you need more power, you can add more machines to each charm without reconfiguring anything else. The load balancer inside the charm knows what to do with the extra help. Ubuntu already offers a number of standard charmsthat are ready to deploy, or you can customize them for your own needs.
Ubuntu Juju is released under the GNU Affero Public License.
Chef
Chef is one of the two big tools used for configuration management. If you like writing straightforward code in Ruby for installing the right packages for your new machines, Chef is the tool for you. You write the instructions for which packages should be installed in which order, then Chef does the work for you. There is a wide variety of plug-ins ready to install many common packages.
Puppet
The other major configuration management tool is Puppet. While many features are the same, the main language for specifying which packages are necessary is a bit different. Puppet asks you for a list of dependencies, then figures out how to install the right packages so that everything is ready to run. Puppet Labs maintains a large collection of plug-ins that simplify many chores.
