veyepar: How to Tame a Data Format

Author(s):	George Wilson
Location	Medium Lecture Theatre CB11.00.401
Date	jan Tue 23	Days Raw Files
Start	13:40	First Raw Start	13:05
Duration	0:45:0	Offset	0:34:03
End	14:25	Last Raw End	14:35
Chapters	00:00 0:25:37
Total cuts_time	44 min.

http://lca2018.linux.org.au/schedule/presentation/150/
raw-playlist raw-mp4-playlist encoded-files-playlist host archive public tweet mp4 svg png

assets release.pdf How_to_Tame_a_Data_Format.json
logs
Admin: episode episode list cut list raw files day marks day marks day image_files

State:

Locked:

clear this to unlock

Locked by:

user/process that locked.

Start:

initially scheduled time from master, adjusted to match reality

Duration:

length in hh:mm:ss

Name:

Video Title (shows in video search results)

Emails:

email(s) of the presenter(s)

Released:

has someone authorised pubication

Normalise:

Channelcopy:

m=mono, 01=copy left to right, 10=right to left, 00=ignore.

Thumbnail:

filename.png

Description:

markdown

Many programmers, data scientists, and other science and mathematics professionals spend a lot of time cleaning and working with data stored in nested formats, such as JSON; or in tabular formats, such as CSV. There are many libraries in many languages for reading and writing these formats.

In strongly statically-typed languages, we achieve a benefit by imposing structure onto the parsed data. The messiness of real data makes this challenging, but hope is not lost!

This talk will discuss design decisions made in existing open source libraries, and highlight the tools and techniques for designing practical and general purely functional libraries to work with these formats, while maintaining the benefits of strong static typing. We will learn about Encode and Decode type classes, zippers, and some different philosophies about error handling. We will see what differences exist in the application of these tools to nested and tabular data formats.

An attendee will leave this talk with an understanding of the design of a modern functional encoding and decoding library. You should be able to jump into one of these libraries and understand how the pieces fit together. New tools will be added to your toolbelt for when you come up against a data format and need to write your own library.

Comment:

production notes