monometric.io RPM repository

monometric.io agent

This repo contains the latest offical monometric.io agent package for CentOS.

You just need to add the .repo-file, and then you can install mm-agent with a simple yum command. Copy and paste this into /etc/yum.repos.d/monometric.repo

[monometric]
name=monometric.io agent
baseurl=https://repo.monometric.io/mm-agent/centos/$basearch
enabled=1
gpgcheck=0      

After that, you can simly run: yum install mm-agent to install mm-agent. You need to configure the agent before it can be run.

You can find your customer id and agent key under your inventory settings. To configure the agent, run
/opt/mm-agent/agent.py --configure --key <secret-agent-key> --customerid <secret-customer-id>
For example:

# /opt/mm-agent/agent.py --configure --key 0132ceb9ec6da1c34a765be3f --customerid 2893dc57a7a9c71b01718cfb
If you install with the script from the monometric.io web interface, your agent will be configured automatically.

Writing plugins

Writing plugins for monometric.io is really easy. We have created a flexible and powerful plugin system, that allows you to write plugins in any language you like. You will literally be up and running in seconds!

Example #1: Getting started

Writing a bash plugin that reports the number of connections we have from or to MySQL:

#!/bin/bash
echo mysql.connections: `netstat -tn | grep ESTABLISHED | grep :3306 | wc -l`
Save this plugin as /etc/mm-agent/plugins/mysql_connections.sh, and make it executable by running
[root@example plugins]# chmod +x /etc/mm-agent/plugins/mysql_connections.sh
Then we test what the output will look like
[root@example plugins]# ./mysql_connections.sh
mysql.connections: 23
[root@example plugins]#
And that's it.

The agent will automatically detect and run our new plugin, no restart required.

We now have a fully funtional, really simple plugin that graphs the number of connections with a source or destionation port of 3306, the MySQL default port. We can graph this value in the monometric.io webinterface, and setup alerts that will notify us if the number drops below three.

If you need to disable the plugin, simply delete it, or run:

[root@example plugins]# chmod -x /etc/mm-agent/plugins/mysql_connections.sh

NOTE: The script will run as an unprivileged user per default, so make sure you actually have access to run the commands in your plugin. If you need to run your plugins as root, simply put them in the /etc/mm-agent/privileged_plugins folder.

Example #2: Another example

Writing a bash plugin to check the ping time to google.com:

#!/bin/bash
echo pingplugin.ms: `ping -c 1 google.com | grep ^rtt |cut -d/ -f 5`
Save this plugin as /etc/mm-agent/plugins/pingplugin.sh, and make it executable by running
[root@example plugins]# chmod +x /etc/mm-agent/plugins/pingplugin.sh
The plugins output will be:
[root@example plugins]# ./pingplugin.sh
pingplugin.ms: 34.231
[root@example plugins]#

The filename doesn't really matter, but the file needs to be executable and readable by the mm-agent user.

Example #3: Running with root privileges and adjusting execution interval

Writing a python script, that scans our access log for unauthorized login attempts:

#!/usr/bin/python
num_lines = 0
unknown_users = 0
fh = open('/var/log/secure')
if fh:
    while True:
        line = fh.readline()
        if not line:
            break
        if line.find('Invalid user ') != -1:
            unknown_users += 1
        num_lines +=1
print "securelog.lines:", num_lines
print "securelog.unknown_users:", unknown_users
In this case, it is important that we put it in /etc/mm-agent/privileged_plugins, as only root can read /var/log/secure on most systems. We verify the output:
[root@example privileged_plugins]# ./scanlog.py
securelog.lines: 2650
securelog.unknown_users: 636
[root@example privileged_plugins]#

The output format is very simple. A line starts with a metric name, followed by colon, and then a value, which can be any string or number. For the key name, only a-z, 0-9 and dots are allowed. A plugin can have several key-value fields as output, separated by a line.

In this example, we scan through our /var/log/secure logfile, and counts lines invalid users. The default plugin execution interval is 10 seconds. As this information isn't too volatile, we can get away with only doing it once every minute, to save resources on our server. To do this, we create a config for our plugin, called scanlog.py.conf:

[plugin]
interval = 60
This way, we instruct the agent to run it at most every 60 seconds. Even if the plugin is very slow, i.e. it takes 70 seconds to finish, the agent will never run more than one instance at once.

Example #4: Working with rates and counters

The python script we just wrote, also reports the number of lines in our logfile. This number isn't very interesting in it self, it is just a counter. What would be more interesting, would be to see the rate of new lines, or rate of invalid login attempts in our logfile. The agent has built in support for this. The only thing we need to do is prefix the key name with _counter.

#!/usr/bin/python
num_lines = 0
unknown_users = 0
fh = open('/var/log/secure')
if fh:
    while True:
        line = fh.readline()
        if not line:
            break
        if line.find('Invalid user ') != -1:
            unknown_users += 1
        num_lines +=1
print "_counter.securelog.lines:", num_lines
print "_counter.securelog.unknown_users:", unknown_users
New output:
[root@root privileged_lugins]# ./scanlog.py
_counter.securelog.lines: 2849
_counter.securelog.unknown_users: 699
[root@root privileged_lugins]#
Now, the agent will automatically detect this, and we can graph this values as value per second. If you would like, you can also multiply the number with 60, to get value per minute by making these small changes to the script:
print "_counter.securelog.lines.per_min:", num_lines * 60
print "_counter.securelog.unknown_users.per_min:", unknown_users * 60

Example #5: Environment variables

If your process needs environment variables, these can be specified in the .conf-file, like the plugin interval. One usecase for this, is code reuse. Let's say we want to ping several hosts, but use the same code. First, we modify our We go back back to example #2, but modify the example slightly.
#!/bin/bash
echo pingplugin.ms.$MY_PLUGIN_METRIC_NAME: `ping -c 1 $MY_PLUGIN_HOST | grep ^rtt |cut -d/ -f 5`
We save this file as pingplugin.sh. We create a config file, pingplugin.sh.conf.
[env]
MY_PLUGIN_METRIC_NAME=google
MY_PLUGIN_HOST=google.com
The variables in pingplugin.sh.conf will be injected in the plugin environment. Testing the plugin can be done like this (or just export the environment variables):
[root@example DO /etc/mm-agent/plugins]# MY_PLUGIN_METRIC_NAME=google MY_PLUGIN_HOST=googl
pingplugin.ms.google: 0.285
[root@example DO /etc/mm-agent/plugins]#

Community

Ultimatly, only your imagination limit what you can monitor from custom plugins. Make sure to identify what is most important for you, and setup alerts. If you are looking for insipiration, head over to our community plugins page (to be announced). Maybe you can find exactly what you are looking for already.