veyepar: collectd in dynamic environments

Author(s):	Florian Forster
Location	OGGB3
Date	jan Wed 14	Days Raw Files
Start	14:15	First Raw Start	14:14
Duration	0:45:00	Offset	0:00:30
End	15:00	Last Raw End	15:09
Chapters	00:00
Total cuts_time	47 min.

http://lca2015.linux.org.au/schedule/30193/view_talk
raw-playlist raw-mp4-playlist encoded-files-playlist host public tweet mp4 svg png

assets release.pdf collectd_in_dynamic_environments.json
logs
Admin: episode episode list cut list raw files day marks day marks day image_files

State:

Locked:

clear this to unlock

Locked by:

user/process that locked.

Start:

initially scheduled time from master, adjusted to match reality

Duration:

length in hh:mm:ss

Name:

Video Title (shows in video search results)

Emails:

email(s) of the presenter(s)

Released:

has someone authorised pubication

Normalise:

Channelcopy:

m=mono, 01=copy left to right, 10=right to left, 00=ignore.

Thumbnail:

filename.png

Description:

markdown

Over the last years, the number of services running "in the cloud", on a dynamic pool of virtual machines, has increased dramatically. While "moving to the cloud" poses an opportunity to cut costs and offers new possibilities with regard to scalability, it also creates new problems for system administrators. Many existing open-source monitoring solutions require a centrally configured list of hosts, for example, which is not possible to specify in a cloud environment. Configuring checks for individual hosts also makes less sense than it used to, and the focus is shifted to detecting a misbehaving site instead of fixing a single host.

collectd is a lightweight daemon which collects, processes and stores system and application performance metrics. It comes with over one hundred plugins for all sorts of systems, applications and devices. The daemon is very light-weight and is in wide-spread use in small devices (e.g. OpenWrt) as well as huge, ever changing cloud setups.

This talk will focus on the "cloud" end of the spectrum. First, it will discuss what makes collectd a good choice for performance data collection in such a dynamic environment. It will discuss querying guest metrics from the hypervisor, to collect some basic information about virtual machines without instrumenting them. We will then look at different ways to set up networking using collectd's "network" plugin. We'll discuss per-site and global data aggregation using collectd's "aggregation" plugin and, alternatively, by using Riemann. At last the talk will cover some common storage systems, from the old-but-proven RRDtool to the newer alternatives, such as Graphite.

Comment:

production notes