Thursday, January 19, 2012

How to Monitor and Log Network Traffic on Linux Using vnStat


This article is part of our on-going performance monitoring series of articles.
In this article, let us focus on how to monitor network traffic and log them for later analysis using vnstat utility.
vnstat is a command line utility that displays and logs network traffic of the interfaces on your systems. This depends on the network statistics provided by the kernel. So, vnstat doesn’t add any additional load to your system for monitoring and logging the network traffic.

1. Install vnStat

Install vnStat on your system from the repository that is specific to your Linux distributions.
For example, on Ubuntu use apt-get to install it as shown below.
$ apt-get install vnstat
If you prefer to install it from source, download vnStat source from here , or use the wget as shown below to download and install it.
Please note that you can also use Mergecap and Tshark to analyze network traffic.
wget http://humdi.net/vnstat/vnstat-1.11.tar.gz
cd vnstat-1.11
make
make install
Please note that you don’t need to do “./configure” that you typically do for other source based installations.
Since vnstat depends on the information provided by kernel, execute the following command to verify whether kernel is providing all the information that vnStat is expecting.
# vnstat --testkernel
This test will take about 60 seconds.
Everything is ok.

2. Pick a Interface to Monitor using vnStat

vnStat doesn’t monitor any interfaces unless you specifically request it to do so.
To start monitoring eth0, do the following. This needs to be executed only once. As you see below, this creates a database file eth0 under /var/lib/vnstat directory that will contain all the network traffic log messages for this specific interface.
# vnstat -u -i eth0
Error: Unable to read database "/var/lib/vnstat/eth0".
Info: -> A new database has been created.
To view all the available interfaces on your system that vnStat can monitor, do the following.
# vnstat --iflist
Available interfaces: lo eth0 eth1 sit0
Start the vnstatd (vnstat daemon), which will monitor and log these information in the background.
# vnstatd -d
# ps -ef | grep vnst
root     14353     1  0 09:12 ?        00:00:00 vnstatd -d
root     14355   330  0 09:12 pts/1    00:00:00 grep vnst
Note: You can add “vnstatd -d” to your /etc/rc.local file, so that it starts automatically anytime you reboot your system.

3. vnStat Basic Usage

vnstat without any argument will give you a quick summary with the following info:
  • The last time when the vnStat datbase located under /var/lib/vnstat/ was updated
  • From when it started collecting the statistics for a specific interface
  • The network statistic data (bytes transmitted, bytes received) for the last two months, and last two days.
# vnstat
Database updated: Sat Oct 15 11:54:00 2011

   eth0 since 10/01/11

          rx:  12.89 MiB      tx:  6.94 MiB      total:  19.82 MiB

   monthly
                     rx      |     tx      |    total    |   avg. rate
     ------------------------+-------------+-------------+---------------
       Sep '11     12.90 MiB |    6.90 MiB |   19.81 MiB |    0.14 kbit/s
       Oct '11     12.89 MiB |    6.94 MiB |   19.82 MiB |    0.15 kbit/s
     ------------------------+-------------+-------------+---------------
     estimated        29 MiB |      14 MiB |      43 MiB |

  daily
                     rx      |     tx      |    total    |   avg. rate
     ------------------------+-------------+-------------+---------------
     yesterday      4.30 MiB |    2.42 MiB |    6.72 MiB |    0.64 kbit/s
         today      2.03 MiB |    1.07 MiB |    3.10 MiB |    0.59 kbit/s
     ------------------------+-------------+-------------+---------------
     estimated         4 MiB |       2 MiB |       6 MiB |
Note: If you just installed the vnStat, it will give the following message “eth0: Not enough data available yet.”. Wait for some time and try the command again.

4. vnStat hours, days, months, weeks Network Data

Use “vnstat -h” (or) “vnstat –hours” for network statistic data breakdown by hour. This also displays a text based graph.
Use “vnstat -d” (or) “vnstat –days” for network statistic data breakdown by day.
# vnstat -d
 eth0  /  daily
         day         rx      |     tx      |    total    |   avg. rate
     ------------------------+-------------+-------------+---------------
      10/10/11      2.48 MiB |    1.28 MiB |    3.76 MiB |    0.36 kbit/s
      10/11/11      4.07 MiB |    2.17 MiB |    6.24 MiB |    0.59 kbit/s
      10/12/11      4.30 MiB |    2.42 MiB |    6.72 MiB |    0.64 kbit/s
      10/13/11      2.06 MiB |    1.10 MiB |    3.16 MiB |    0.60 kbit/s
     ------------------------+-------------+-------------+---------------
     estimated         3 MiB |       1 MiB |       4 MiB |
Use “vnstat -m” (or) “vnstat –months” for network statistic data breakdown by month.
# vnstat --m

 eth0  /  monthly

       month        rx      |     tx      |    total    |   avg. rate
    ------------------------+-------------+-------------+---------------
      Sep '11     12.90 MiB |    6.90 MiB |   19.81 MiB |    0.14 kbit/s
      Oct '11     12.92 MiB |    6.96 MiB |   19.89 MiB |    0.15 kbit/s
    ------------------------+-------------+-------------+---------------
    estimated        29 MiB |      14 MiB |      43 MiB |
Similar to days and months, use “vnstat -m” (or) “vnstat –months” for network statistic data breakdown by week.

5. Export the data to Excel or other DB

If you like to export the network monitoring data to an excel or other database, you can dump the data in a text format delimited with semi-colon, which you can import to Excel or other db.
The 1st few lines of the –dumpdb output contains some header information. After the header lines, it has 30 lines that starts with “d;” (d;0;1318316406;1;0;386;698;1). This lines has the following information separated by semi-colon.
  • d – stands for days
  • 0 – number of the day. 0 indicates today.
  • 1318316406 – data in Unix format
  • Followed by this, it contains the bytes transmitted and received
$ vnstat --dumpdb
interface;eth0
created;1218562937
updated;1218546895
totalrx;3
totaltx;1
...
...
d;0;1328316406;1;0;386;698;1
d;1;1345262937;2;1;494;289;1
You can also use “vnstat –oneline”, which displays the traffic summary in a single line where the values are delimited with semi-colon.
$ vnstat --oneline
1;eth0;10/11/11;1.45 MiB;801 KiB;2.23 MiB;0.59 kbit/s;Oct '11;3.93 MiB;2.06 MiB;6.00 MiB;0.05 kbit/s;3.93 MiB;2.06 MiB;6.00 MiB

6. Display Live Network Statistics

Use “vnstat -l” or “vnstat –live” to display the live network statistic information.
$ vnstat -l
Monitoring eth0...    (press CTRL-C to stop)

   rx:        2 kbit/s     5 p/s          tx:        2 kbit/s     4 p/s
After you press Ctrl-C to stop it, vnstat will display a summary for the time period the live monitor was running.

7. Change the default vnstat output format

Use “vnstat -s” or “vnstat –short” which will display a short summary of the network statistics. This includes statistics for today, yesterday and for the current month.
$ vnstat -s (--short)

                      rx      /      tx      /     total    /   estimated
 eth0:
       Oct '11      3.93 MiB  /    2.06 MiB  /    6.00 MiB  /   13.00 MiB
     yesterday      2.48 MiB  /    1.28 MiB  /    3.76 MiB
         today      1.45 MiB  /     801 KiB  /    2.23 MiB  /      --
You can also use “vnstat –style 0″, which will give a narrow column output, which is easier to read than the default wider column output.
$ vnstat --style 0
Following are the available style number:
  • 0 – Narrow output
  • 1 – Enable bar column
  • 2 – Enable bar column, and displays average traffic rate in summary
  • 3 – Display average traffic rate in all output
  • 4 – when combined with live mode (vnstat -l), disable the terminal control characters

8. Display Top 10 Traffic Days

Use “vnstat -t” or “vnstat –top10″ to display all time top 10 traffic days.
$ vnstat --top10

 eth0  /  top 10

    #      day          rx      |     tx      |    total    |   avg. rate
   -----------------------------+-------------+-------------+---------------
    1   10/12/11       4.30 MiB |    2.42 MiB |    6.72 MiB |    0.64 kbit/s
    2   10/11/11       4.07 MiB |    2.17 MiB |    6.24 MiB |    0.59 kbit/s
    3   10/10/11       2.48 MiB |    1.28 MiB |    3.76 MiB |    0.36 kbit/s
    ....
   -----------------------------+-------------+-------------+---------------