veyepar: 400,000 ephemeral containers: testing entire ecosystems with Docker

Author(s):	Daniel Axtens
Location	Tasman Hall A
Date	jan Tue 17	Days Raw Files
Start	20:20	First Raw Start	19:48
Duration	0:45:0	Offset	0:31:31
End	21:05	Last Raw End	21:18
Chapters	00:00 0:26:59
Total cuts_time	43 min.

http://linux.conf.au/schedule/presentation/81/
raw-playlist raw-mp4-playlist encoded-files-playlist host archive tweet mp4 svg png

assets release.pdf 400000_ephemeral_containers_testing_entire_ecosystems_with_Docker.json
logs
Admin: episode episode list cut list raw files day marks day marks day image_files

State:
Locked:	clear this to unlock
Locked by:	user/process that locked.
Start:	initially scheduled time from master, adjusted to match reality
Duration:	length in hh:mm:ss
Name:	Video Title (shows in video search results)
Emails:	email(s) of the presenter(s)
Released:	has someone authorised pubication
Normalise:
Channelcopy:	m=mono, 01=copy left to right, 10=right to left, 00=ignore.
Thumbnail:	filename.png
Description:	The value of a high-level language isn't just in the language itself, but rather in the ecosystem surrounding it. Python is hugely enriched by PyPI, Ruby by gems, and Node.js by npm. These ecosystems have hundreds of thousands of packages, evolve rapidly, and vary wildly in quality. What if you want to test how _all_ the packages work in a given situation? This is the question we set out to answer in this talk. Previously, testing each and every one of the 458,100+ packages came with an enormous infrastructure cost - for each package you need a VM with a full OS, and the associated memory, CPU and disk requirements. This makes exhaustive testing infeasible for most teams. However, we found that by using Docker, we could test the entire Python, Ruby or Node.js ecosystem over a weekend with a single VM. In this talk, we look at: * how we leveraged Docker to rapidly test the entire Python, Ruby and Node.js package sets on OpenPOWER, * how the characteristics of containers vs VMs made an otherwise infeasible task both feasible and simple, * how we evolved this into a continuous-integration process, * other interesting applications of testing at this scale, * what we found, and some of the interesting quirks and bugs we encountered and lessons we learned. markdown
Comment:	production notes