Hi
user
Admin Login:
Username:
Password:
Name:
The Evolution of Python Packaging at Facebook
--client
pyconau
--show
pycon_au_2016
--room Room_103 11358 --force
Next: 1 Preventing Cat-astrophes with GNU MediaGoblin
show more...
Marks
Author(s):
Dan Reif
Location
Room 103
Date
aug Sun 14
Days Raw Files
Start
14:20
First Raw Start
error-in-template
Duration
0:40:00
Offset
None
End
15:00
Last Raw End
Chapters
Total cuts_time
None min.
https://2016.pycon-au.org/schedule/131/view_talk
raw-playlist
raw-mp4-playlist
encoded-files-playlist
mp4
svg
png
assets
release.pdf
The_Evolution_of_Python_Packaging_at_Facebook.json
logs
Admin:
episode
episode list
cut list
raw files day
marks day
marks day
image_files
State:
---------
borked
edit
encode
push to queue
post
richard
review 1
email
review 2
make public
tweet
to-miror
conf
done
Locked:
clear this to unlock
Locked by:
user/process that locked.
Start:
initially scheduled time from master, adjusted to match reality
Duration:
length in hh:mm:ss
Name:
Video Title (shows in video search results)
Emails:
email(s) of the presenter(s)
Released:
Unknown
Yes
No
has someone authorised pubication
Normalise:
Channelcopy:
m=mono, 01=copy left to right, 10=right to left, 00=ignore.
Thumbnail:
filename.png
Description:
With tens of millions of Python invocations every day, what’s a packager to do? The answer selected must account for the insanity of a deeply-heterogeneous production environment: different kernels, different OS distributions, even different versions of system Python. It also needs enough context to carry with it a consistent reference point for when it was packaged relative to the work of thousands of engineers in a single, unified source control tree. Lastly, at Facebook scale with hundreds of thousands of servers, every byte sent over the network and stored on disk counts, and every CPU cycle wasted can create a myriad of challenges associated with data center operations management. Sure, it’d be easy to show what a beautiful, easy packaging format we’ve developed at Facebook, and sing its praises, but that’s not what this talk is about. Instead, we’ll get into the nitty-gritty, and talk about hard tradeoffs that happen when developing a system in the real world. This is an in-depth look at how Facebook’s packaging has evolved, warts and speed bumps included. Some of the design goals we addressed (and/or issues we hit!) included: * Synchronizing and pinning versioning of underlying compiled libraries across related tools * Running the package as a transparently-Pythonic command-line utility, à la shebang, so we could do in-place replacements of packages written in other languages (e.g. C++) * Optimizing package size transferred across the network * Optimizing package sizes on disk * Minimizing package launch times * Handling packages launched from a network-FS location that subsequently goes offline Attendees will learn about the different tools and techniques we used to solve these challenges, as well as the reasoning behind any trade offs that were made.
markdown
Comment:
production notes
Rf filename:
root is .../show/dv/location/, example: 2013-03-13/13:13:30.dv
Sequence:
get this:
check and save to add this
Veyepar
Video Eyeball Processor and Review