Metrics Cocktail: StatsD+InfluxDB+Grafana

Collecting and analyzing metrics is a key to understand a system’s runtime behaviors and thus to provide a better guideline of optimization. As the user of a metrics system, you might want to collect metric and event with simple APIs, retrieve historical metric data. and view visualized metrics statistics. This article shows an example metrics system that composes of a client, a datastore, and visualization UI. There are many available open source packages. This article shows you how to build a metric system with StatsD, influxdb, and Grafana.

The figure above shows you the data flow. User sends application metrics to StatsD. StatsD stores the metrics data in InfluxDB, a time-series database. Users can view visualized metrics from Grafana.

Let’s briefly talk about the softwares. StatsD is a daemon that collects and aggregates metrics data. Its UDP protocol is well accepted. There are clients library for almost all major programming language. StatsD can also acts as  a local aggregator. StatsD also provide a extensible backend architecture. Grafana is a metrics dashboard that provides data visualization and generates statistic graph. InfluxDB is a time-seies database. 

The rest of this article shows you how to installation and configuration steps in Ubuntu.

InfluxDB

The installation is pretty simple. Below summaries the steps from InfluxDB’s website. You can see the full instruction here.

wget http://s3.amazonaws.com/influxdb/influxdb_latest_i386.deb
sudo dpkg -i influxdb_latest_i386.deb
sudo /etc/init.d/influxdb start

You should be able to access to the web UI at port 8083(http://localhost:8083/).

Enter the username root and password root and click Connect. You’ll then see a logged in screen like this:

Create a ‘demo’ database!

StatsD

Install nodejs, npm, statsd, and InfluxDB backend.

sudo apt-get install nodejs
cd /opt
sudo git clone https://github.com/etsy/statsd.git
cd statsd
apt-get install npm
npm install statsd-influxdb-backend -d

Edit /opt/statsd/config.js to configure StatsD to send data to InfluxDB.
 

{
    influxdb: {
        host: '127.0.0.1', // InfluxDB host. (default 127.0.0.1)
        port: 8086, // InfluxDB port. (default 8086)
        database: ‘demo ',  // InfluxDB database instance. (required)
        username: 'root', // InfluxDB database username. (required)
        password: 'root', // InfluxDB database password. (required)
        flush: {
            enable: true // Enable regular flush strategy. (default true)
        },
        proxy: {
            enable: false, // Enable the proxy strategy. (default false)
            suffix: 'raw', // Metric name suffix. (default 'raw')
            flushInterval: 1000 // Flush interval for the internal buffer.
                // (default 1000)
        }
    },
    port: 8125, // StatsD port.
    backends: ['./backends/console’, 'statsd-influxdb-backend'],
    debug: true,
    legacyNamespace: false
}

Start Statsd

nodejs stats.js config.js

Grafana

Installation

apt-get install apache2
wget http://grafanarel.s3.amazonaws.com/grafana-1.5.4.tar.gz
tar xvf grafana-1.5.4.tar.gz
mv grafana-1.5.4/ /var/www/
cd /var/www/grafana-1.5.4
cp config.sample.js config.js

Configure Grafana to use InfluxDB. Edit /var/www/grafana-1.5.4/config.js

// datasources, you can add multiple
datasources: {
        influxdb: {
            type: 'influxdb',
            url: "http://<influxdb-host>:8086/db/demo",
            default: true,
            username: 'root',
            password: 'root'
        },

Access Grafana at http://<grafana-host>/grafana-1.5.4/.
Learn to use Grafana interface, visit the beginner guide.

Send some data ! 

Here is an example that uses the StatsD Python Client. This example sends random counter to StatsD.

Install  Python StatsD library. 

sudo pip install statsd

Create test.py

#!/usr/bin/python
import time
import random
import statsd
counter_name = ‘my.test.1'
wait_s = 1
while 1:
    c = statsd.StatsClient('localhost', 8125)
    random_count = random.randrange(1, 100)
    print 'Count=(%d)' % random_count
    while random_count > 0:
        c.incr(counter_name, 5)
        random_count -= 1
    time.sleep(wait_s)

Execute test.py

python test.py

You should be able to see your data from Grafana now.
 

Reference:

• http://influxdb.com/docs/v0.7/introduction/getting_started.html
• http://influxdb.com/docs/v0.7/api/query_language.html
• https://github.com/etsy/statsd/
• http://grafana.org/docs/