veyepar: Monitoring All the Things! on your Linux system with the Elastic Stack

Author(s):	Josh Rich
Location	Great Hall CB01.05.009
Date	jan Tue 23	Days Raw Files
Start	14:05	First Raw Start	13:54
Duration	0:25:0	Offset	0:10:51
End	14:30	Last Raw End	14:54
Chapters	00:00 0:18:34 0:20:55
Total cuts_time	24 min.

http://lca2018.linux.org.au/schedule/presentation/161/
raw-playlist raw-mp4-playlist encoded-files-playlist host archive public tweet mp4 svg png

assets release.pdf Monitoring_All_the_Things_on_your_Linux_system_with_the_Elastic_Stack.json
logs
Admin: episode episode list cut list raw files day marks day marks day image_files

State:

Locked:

clear this to unlock

Locked by:

user/process that locked.

Start:

initially scheduled time from master, adjusted to match reality

Duration:

length in hh:mm:ss

Name:

Video Title (shows in video search results)

Emails:

email(s) of the presenter(s)

Released:

has someone authorised pubication

Normalise:

Channelcopy:

m=mono, 01=copy left to right, 10=right to left, 00=ignore.

Thumbnail:

filename.png

Description:

markdown

In this talk, we'll look at how you can easily ingest your Linux system logs and various OS metrics into Elasticsearch using Filebeat and Metricbeat modules.  Modules are a new concept in the open-source Filebeat and Metricbeat tools made by Elastic. We can then visually examine both our systems performance and all events occurring on it over time with Kibana. This is a near complete open source monitoring solution for a Linux system.

Assuming the demo gods allow, We'll have a little bit of a play with our systems, by inducing CPU/memory load or spamming log lines to see it reacts in Kibana, and correlate the different sources of information together in a single Kibana dashboard, providing a relatively complete view of what is happening on the system.

Finally, anything missing we want to monitor or record we can do by writing our own Filebeat or Metricbeat module. So we will take a dive into the code to see how you can contribute your own Filebeat or Metricbeat module to these projects.

Comment:

production notes

2018-01-23/13_54_09.ts Apply: 13:54:09 - 14:05:26 ( 00:11:17 ) S: 13:54:09 - E: 14:24:09 D: 00:30:00 (End: 677.0) show more...

vlc ~/Videos/veyepar/lca/lca2018/dv/greathall/2018-01-23/13_54_09.ts :start-time=00.0 --audio-desync=0

	Raw File	Cut List
	13:54:09		seconds: 0.0 Wall: 13:54:09
Duration	00:30:00
	14:24:09		seconds: 677.0 Wall: 14:05:26
Comments:
mp4 mp4.m3u	dv.m3u	Split: Sequence:: delete

2018-01-23/13_54_09.ts Apply: 14:05:26 - 14:05:35 ( 00:00:09 ) S: 13:54:09 - E: 14:24:09 D: 00:30:00 (Start: 677.0) (End: 686.0) show more...

vlc ~/Videos/veyepar/lca/lca2018/dv/greathall/2018-01-23/13_54_09.ts :start-time=0677.0 --audio-desync=0

	Raw File	Cut List
	13:54:09		seconds: 677.0 Wall: 14:05:26
Duration	00:30:00
	14:24:09		seconds: 686.0 Wall: 14:05:35
Comments:
mp4 mp4.m3u	dv.m3u	Split: Sequence:: delete

2018-01-23/13_54_09.ts Apply: 14:05:35 - 14:24:09 ( 00:18:34 ) S: 13:54:09 - E: 14:24:09 D: 00:30:00 (Start: 686.0) show more...

vlc ~/Videos/veyepar/lca/lca2018/dv/greathall/2018-01-23/13_54_09.ts :start-time=0686.0 --audio-desync=0

	Raw File	Cut List
	13:54:09		seconds: 686.0 Wall: 14:05:35
Duration	00:30:00
	14:24:09		seconds: 0.0 Wall: 13:54:09
Comments:
mp4 mp4.m3u	dv.m3u	Split: Sequence:: delete

2018-01-23/14_24_09.ts Apply: 14:24:09 - 14:26:30 ( 00:02:21 ) S: 14:24:09 - E: 14:54:09 D: 00:30:00 (End: 141.0) show more...

vlc ~/Videos/veyepar/lca/lca2018/dv/greathall/2018-01-23/14_24_09.ts :start-time=00.0 --audio-desync=0

	Raw File	Cut List
	14:24:09		seconds: 0.0 Wall: 14:24:09
Duration	00:30:00
	14:54:09		seconds: 141.0 Wall: 14:26:30
Comments:
mp4 mp4.m3u	dv.m3u	Split: Sequence:: delete

2018-01-23/14_24_09.ts Apply: 14:26:30 - 14:30:07 ( 00:03:37 ) S: 14:24:09 - E: 14:54:09 D: 00:30:00 (Start: 141.0) (End: 358.0) show more...

vlc ~/Videos/veyepar/lca/lca2018/dv/greathall/2018-01-23/14_24_09.ts :start-time=0141.0 --audio-desync=0

	Raw File	Cut List
	14:24:09		seconds: 141.0 Wall: 14:26:30
Duration	00:30:00
	14:54:09		seconds: 358.0 Wall: 14:30:07
Comments:
mp4 mp4.m3u	dv.m3u	Split: Sequence:: delete

2018-01-23/14_24_09.ts Apply: 14:30:07 - 14:34:39 ( 00:04:32 ) S: 14:24:09 - E: 14:54:09 D: 00:30:00 (Start: 358.0) (End: 630.0) show more...

vlc ~/Videos/veyepar/lca/lca2018/dv/greathall/2018-01-23/14_24_09.ts :start-time=0358.0 --audio-desync=0

	Raw File	Cut List
	14:24:09		seconds: 358.0 Wall: 14:30:07
Duration	00:30:00
	14:54:09		seconds: 630.0 Wall: 14:34:39
Comments:
mp4 mp4.m3u	dv.m3u	Split: Sequence:: delete

2018-01-23/14_24_09.ts Apply: 14:34:39 - 14:50:51 ( 00:16:12 ) S: 14:24:09 - E: 14:54:09 D: 00:30:00 (Start: 630.0) (End: 1602.0) show more...

vlc ~/Videos/veyepar/lca/lca2018/dv/greathall/2018-01-23/14_24_09.ts :start-time=0630.0 --audio-desync=0

	Raw File	Cut List
	14:24:09		seconds: 630.0 Wall: 14:34:39
Duration	00:30:00
	14:54:09		seconds: 1602.0 Wall: 14:50:51
Comments:
mp4 mp4.m3u	dv.m3u	Split: Sequence:: delete

2018-01-23/14_24_09.ts Apply: 14:50:51 - 14:51:27 ( 00:00:36 ) S: 14:24:09 - E: 14:54:09 D: 00:30:00 (Start: 1602.0) (End: 1638.0) show more...

vlc ~/Videos/veyepar/lca/lca2018/dv/greathall/2018-01-23/14_24_09.ts :start-time=01602.0 --audio-desync=0

	Raw File	Cut List
	14:24:09		seconds: 1602.0 Wall: 14:50:51
Duration	00:30:00
	14:54:09		seconds: 1638.0 Wall: 14:51:27
Comments:
mp4 mp4.m3u	dv.m3u	Split: Sequence:: delete

2018-01-23/14_24_09.ts Apply: 14:51:27 - 14:54:09 ( 00:02:42 ) S: 14:24:09 - E: 14:54:09 D: 00:30:00 (Start: 1638.0) show more...

vlc ~/Videos/veyepar/lca/lca2018/dv/greathall/2018-01-23/14_24_09.ts :start-time=01638.0 --audio-desync=0

	Raw File	Cut List
	14:24:09		seconds: 1638.0 Wall: 14:51:27
Duration	00:30:00
	14:54:09		seconds: 0.0 Wall: 14:24:09
Comments:
mp4 mp4.m3u	dv.m3u	Split: Sequence:: delete