veyepar: Experiences debugging memory bloat and high CPU consumption in python

Author(s):	Alexandre Hardy
Location	Orange Room
Date	oct Fri 07	Days Raw Files
Start	10:00	First Raw Start	error-in-template
Duration	00:45:00	Offset	None
End	10:45	Last Raw End
Chapters
Total cuts_time	None min.

https://za.pycon.org/talks/23/
raw-playlist raw-mp4-playlist encoded-files-playlist mp4 svg png

assets release.pdf Experiences_debugging_memory_bloat_and_high_CPU_consumption_in_python.json
logs
Admin: episode episode list cut list raw files day marks day marks day image_files

State:

Locked:

clear this to unlock

Locked by:

user/process that locked.

Start:

initially scheduled time from master, adjusted to match reality

Duration:

length in hh:mm:ss

Name:

Video Title (shows in video search results)

Emails:

email(s) of the presenter(s)

Released:

has someone authorised pubication

Normalise:

Channelcopy:

m=mono, 01=copy left to right, 10=right to left, 00=ignore.

Thumbnail:

filename.png

Description:

markdown

Abstract:

This talk is targeted at python developers who develop long running services, which are susceptible to memory issues or unacceptable CPU usage (as determined by the developer or operations teams).  We focus on debugging techniques that we have used in constrained environments (production like environments) where installation of additional software packages is not permitted, and techniques which we were able to use to debug a python process without restarting it (to avoid state loss).

Topics covered in the talk:
   1. An example debugging session in which memory bloat is diagnosed.
   2. An example debugging session in which high CPU usage is diagnosed.
   3. Tools which can be used in debugging these issues, and their limitations.
   4. Some thoughts about improving how we think about python, and assumptions made by python developers.
   5. Projects we are (slowly) working on to help us debug python processes in production, or production like environments.

The audience should learn about some useful standard python libraries which can be used to debug memory and CPU usage related issues, and how they were effectively used to solve problems in an enterprise product. The audience will also be encouraged to think differently about python programming, and encouraged to think about what it means to select python (or any other programming language) for a given task.

Comment:

production notes