veyepar: (re)implementing (Google) maglev in Rust

Hi user

Admin Login:

Name: (re)implementing (Google) maglev in Rust
--client lca --show lca2018 --room guthrie 13573 --force
Next: 14 Make a bigger pie: Updating the partition count in a consistent hash ring

show more... Marks

Author(s):	Robert Collins
Location	Guthrie Theatre CB06.03.28
Date	jan Wed 24	Days Raw Files
Start	11:40	First Raw Start	11:10
Duration	0:45:0	Offset	0:29:41
End	12:25	Last Raw End	12:40
Chapters	00:00 0:29:28
Total cuts_time	45 min.

http://lca2018.linux.org.au/schedule/presentation/77/
raw-playlist raw-mp4-playlist encoded-files-playlist host archive public tweet mp4 svg png

final audio

assets release.pdf reimplementing_Google_maglev_in_Rust.json
logs
Admin: episode episode list cut list raw files day marks day marks day image_files

State:
Locked:	clear this to unlock
Locked by:	user/process that locked.
Start:	initially scheduled time from master, adjusted to match reality
Duration:	length in hh:mm:ss
Name:	Video Title (shows in video search results)
Emails:	email(s) of the presenter(s)
Released:	has someone authorised pubication
Normalise:
Channelcopy:	m=mono, 01=copy left to right, 10=right to left, 00=ignore.
Thumbnail:	filename.png
Description:	A war story: Developer re-implements a Google whitepaper in Rust using netmap for networking. In 2016 Google released a whitepaper about their small packet wire rate - 40Gbps even, highly available IP load balancer. I happened to have some weeks shortly after that happened where I was able to do whatever I wanted... and I decided to have some fun and implement an open source look-alike. Maglev has some distinctive design choices - such as the use of userspace networking to achieve zero-copy forwarding - which I knew about but had not actively used, as well as doing direct server return - sending packets to backend servers wrapped in GRE, but traffic in the other direction is routed directly, so the traffic pattern is asymmetric. Since I was young and foolish then, I thought it would be good to also deepen my knowledge of the Rust language at the same time... and develop it on my not-as-Linux-friendly-as-I'd-like Surface : so my Linux environment was actually a Hyper-V VM. This combination led to discovering some truely fantastic facilities in Rust, bugs in the Linux kernel and the netmap project, and exploring some of the awkwardness where very clever C datastructures don't map well to Rust's ownership model. The current status of the project is that the core use case works: traffic can be forwarded using network ring buffers, without needing a dedicated network card, routed with zerocopy while preserving GRE wrappers. Higher level facilities like dynamic reconfiguration and backend health signalling are not yet implemented (but are designed). In the talk I will cover the choices I made and why - rust vs the world, netmap vs DPDK, things I found surprising about doing userspace networking, detail the gritty details of doing userspace networking with Netmap from Rust, using Rust to parse and create network packets, as well as driving netlink from within Rust. I'll cover the bugs I had to fix in order to write the code I wanted to write. markdown
Comment:	production notes

recording sheet

recording sheet

2018-01-24/11_10_19.ts Apply: 11:10:19 - 11:20:03 ( 00:09:44 ) S: 11:10:19 - E: 11:40:18 D: 00:29:59 (End: 584.0) show more...

vlc ~/Videos/veyepar/lca/lca2018/dv/guthrie/2018-01-24/11_10_19.ts :start-time=00.0 --audio-desync=0

	Raw File	Cut List
	11:10:19		seconds: 0.0 Wall: 11:10:19
Duration	00:29:59
	11:40:18		seconds: 584.0 Wall: 11:20:03
Comments:
mp4 mp4.m3u	dv.m3u	Split: Sequence:: delete

audio viz

2018-01-24/11_10_19.ts Apply: 11:20:03 - 11:40:18 ( 00:20:15 ) S: 11:10:19 - E: 11:40:18 D: 00:29:59 (Start: 584.0) show more...

vlc ~/Videos/veyepar/lca/lca2018/dv/guthrie/2018-01-24/11_10_19.ts :start-time=0584.0 --audio-desync=0

	Raw File	Cut List
	11:10:19		seconds: 584.0 Wall: 11:20:03
Duration	00:29:59
	11:40:18		seconds: 0.0 Wall: 11:10:19
Comments:
mp4 mp4.m3u	dv.m3u	Split: Sequence:: delete

audio viz

2018-01-24/11_40_19.ts Apply: 11:40:19 - 11:40:50 ( 00:00:31 ) S: 11:40:19 - E: 12:10:18 D: 00:29:59 (End: 31.0) show more...

vlc ~/Videos/veyepar/lca/lca2018/dv/guthrie/2018-01-24/11_40_19.ts :start-time=00.0 --audio-desync=0

	Raw File	Cut List
	11:40:19		seconds: 0.0 Wall: 11:40:19
Duration	00:29:59
	12:10:18		seconds: 31.0 Wall: 11:40:50
Comments:
mp4 mp4.m3u	dv.m3u	Split: Sequence:: delete

audio viz

2018-01-24/11_40_19.ts Apply: 11:40:50 - 12:10:18 ( 00:29:28 ) S: 11:40:19 - E: 12:10:18 D: 00:29:59 (Start: 31.0) show more...

vlc ~/Videos/veyepar/lca/lca2018/dv/guthrie/2018-01-24/11_40_19.ts :start-time=031.0 --audio-desync=0

	Raw File	Cut List
	11:40:19		seconds: 31.0 Wall: 11:40:50
Duration	00:29:59
	12:10:18		seconds: 0.0 Wall: 11:40:19
Comments:
mp4 mp4.m3u	dv.m3u	Split: Sequence:: delete

audio viz

2018-01-24/12_10_19.ts Apply: 12:10:19 - 12:26:49 ( 00:16:30 ) S: 12:10:19 - E: 12:40:19 D: 00:30:00 (End: 990.0) show more...

vlc ~/Videos/veyepar/lca/lca2018/dv/guthrie/2018-01-24/12_10_19.ts :start-time=00.0 --audio-desync=0

	Raw File	Cut List
	12:10:19		seconds: 0.0 Wall: 12:10:19
Duration	00:30:00
	12:40:19		seconds: 990.0 Wall: 12:26:49
Comments:
mp4 mp4.m3u	dv.m3u	Split: Sequence:: delete

audio viz

2018-01-24/12_10_19.ts Apply: 12:26:49 - 12:40:19 ( 00:13:30 ) S: 12:10:19 - E: 12:40:19 D: 00:30:00 (Start: 990.0) show more...

vlc ~/Videos/veyepar/lca/lca2018/dv/guthrie/2018-01-24/12_10_19.ts :start-time=0990.0 --audio-desync=0

	Raw File	Cut List
	12:10:19		seconds: 990.0 Wall: 12:26:49
Duration	00:30:00
	12:40:19		seconds: 0.0 Wall: 12:10:19
Comments:
mp4 mp4.m3u	dv.m3u	Split: Sequence:: delete

audio viz

Rf filename:	root is .../show/dv/location/, example: 2013-03-13/13:13:30.dv
Sequence:
get this:	check and save to add this

2018-01-24/11_10_19.ts
audio viz

audio viz

2018-01-24/11_40_19.ts
audio viz

audio viz

2018-01-24/12_10_19.ts
audio viz

audio viz

Veyepar Video Eyeball Processor and Review