veyepar: Pyrobuf and the Tyranny of Speed

Author(s):	Gregory Saunders
Location	Room 106
Date	aug Sun 14	Days Raw Files
Start	11:10	First Raw Start	error-in-template
Duration	0:40:00	Offset	None
End	11:50	Last Raw End
Chapters
Total cuts_time	None min.

https://2016.pycon-au.org/schedule/171/view_talk
raw-playlist raw-mp4-playlist encoded-files-playlist mp4 svg png

assets release.pdf Pyrobuf_and_the_Tyranny_of_Speed.json
logs
Admin: episode episode list cut list raw files day marks day marks day image_files

State:

Locked:

clear this to unlock

Locked by:

user/process that locked.

Start:

initially scheduled time from master, adjusted to match reality

Duration:

length in hh:mm:ss

Name:

Video Title (shows in video search results)

Emails:

email(s) of the presenter(s)

Released:

has someone authorised pubication

Normalise:

Channelcopy:

m=mono, 01=copy left to right, 10=right to left, 00=ignore.

Thumbnail:

filename.png

Description:

markdown

The data structures used to represent information and program state in memory are rarely conducive to the storage and transmission of that information and state. The process of translating those data structures into a format better suited to storage is known as serialisation or marshalling.

Google’s Protobuf is a mechanism for serialising and deserialising structured data. Pyrobuf is a alternative, lightning-fast, implementation of Protobuf written in Cython that is 2-4 times faster than Protobuf when using its C++ backend and 20-40 times faster than its pure Python implementation.

In this talk we’ll briefly review Protobuf and compare it to other well known serialisation mechanisms such as XML and JSON. We’ll then take a look at Pyrobuf and the basic strategies it uses to achieve significantly better performance.

Pyrobuf’s main claim to fame is its speed. This makes fixing bugs and adding new features a real challenge as any degradation in performance will not be well-received by users who’ve come to expect and rely on its speed. Therefore, for the remainder of the talk we’ll discuss tricks and tips, some more legitimate than others, for maintaining speed as new features are added.

Comment:

production notes