veyepar: Python Profiling and Performance Tuning in Production

State:
Locked:	clear this to unlock
Locked by:	user/process that locked.
Start:	initially scheduled time from master, adjusted to match reality
Duration:	length in hh:mm:ss
Name:	Video Title (shows in video search results)
Emails:	email(s) of the presenter(s)
Released:	has someone authorised pubication
Normalise:
Channelcopy:	m=mono, 01=copy left to right, 10=right to left, 00=ignore.
Thumbnail:	filename.png
Description:	Pinterest decreased latency and shrunk their front-end fleet by over 40% with less than 100 lines of python. This was done by understanding how a significant amount of latency and consumed resources were the result of gevent’s cooperative multitasking. 3 different open source python profilers, were then used together to identify and fix bottlenecks and inefficiencies in the code base. Most profilers are too slow to use in production, so profiling of web servers often occurs in synthetic environments. But to get an accurate picture of what is happening you need production profiling data, so a production ready Linux signal (SIGVTALRM) based python profiler was used. This talk will cover how Pinterest shrunk their front-end fleet, the techniques and tooling used to improve performance and how they can be applied elsewhere. markdown
Comment:	production notes

2017-01-19/13_58_39.ts Apply: 20:58:39 - 21:06:06 ( 00:07:27 ) S: 20:58:39 - E: 21:28:39 D: 00:30:00 (End: 447.0) show more...

vlc ~/Videos/veyepar/lca/lca2017/dv/tasman_bc/2017-01-19/13_58_39.ts :start-time=00.0 --audio-desync=0

2017-01-19/13_58_39.ts Apply: 21:06:06 - 21:12:52 ( 00:06:46 ) S: 20:58:39 - E: 21:28:39 D: 00:30:00 (Start: 447.0) (End: 853.0) show more...

vlc ~/Videos/veyepar/lca/lca2017/dv/tasman_bc/2017-01-19/13_58_39.ts :start-time=0447.0 --audio-desync=0

2017-01-19/13_58_39.ts Apply: 21:12:52 - 21:14:55 ( 00:02:03 ) S: 20:58:39 - E: 21:28:39 D: 00:30:00 (Start: 853.0) (End: 976.0) show more...

vlc ~/Videos/veyepar/lca/lca2017/dv/tasman_bc/2017-01-19/13_58_39.ts :start-time=0853.0 --audio-desync=0

2017-01-19/13_58_39.ts Apply: 21:14:55 - 21:15:08 ( 00:00:13 ) S: 20:58:39 - E: 21:28:39 D: 00:30:00 (Start: 976.0) (End: 989.0) show more...

vlc ~/Videos/veyepar/lca/lca2017/dv/tasman_bc/2017-01-19/13_58_39.ts :start-time=0976.0 --audio-desync=0

2017-01-19/13_58_39.ts Apply: 21:15:08 - 21:28:39 ( 00:13:31 ) S: 20:58:39 - E: 21:28:39 D: 00:30:00 (Start: 989.0) show more...

vlc ~/Videos/veyepar/lca/lca2017/dv/tasman_bc/2017-01-19/13_58_39.ts :start-time=0989.0 --audio-desync=0

2017-01-19/14_28_39.ts Apply: 21:28:39 - 21:49:31 ( 00:20:52 ) S: 21:28:39 - E: 21:58:38 D: 00:29:59 (End: 1252.0) show more...

vlc ~/Videos/veyepar/lca/lca2017/dv/tasman_bc/2017-01-19/14_28_39.ts :start-time=00.0 --audio-desync=0

2017-01-19/14_28_39.ts Apply: 21:49:31 - 21:58:38 ( 00:09:07 ) S: 21:28:39 - E: 21:58:38 D: 00:29:59 (Start: 1252.0) show more...

vlc ~/Videos/veyepar/lca/lca2017/dv/tasman_bc/2017-01-19/14_28_39.ts :start-time=01252.0 --audio-desync=0

2017-01-19/14_58_39.ts Apply: 21:58:39 - 22:28:38 ( 00:29:59 ) S: 21:58:39 - E: 22:28:38 D: 00:29:59 show more...

vlc ~/Videos/veyepar/lca/lca2017/dv/tasman_bc/2017-01-19/14_58_39.ts :start-time=00.0 --audio-desync=0

Rf filename:	root is .../show/dv/location/, example: 2013-03-13/13:13:30.dv
Sequence:
get this:	check and save to add this