Hi
user
Admin Login:
Username:
Password:
Name:
Big Data Deduplication and data matching using Python
--client
pyconau
--show
pycon_au_2013
--room Tasman_B 2527 --force
Next: 10 Managing scientific simulations with Python with RQ (Redis Queue)
show more...
Marks
Author(s):
Andrew Rowe
Location
Tasman B
Date
jul Sun 07
Days Raw Files
Start
14:20
First Raw Start
14:06
Duration
0:30:00
Offset
0:13:21
End
14:50
Last Raw End
15:36
Chapters
00:00
0:24:01
Total cuts_time
29 min.
http://2013.pycon-au.org/schedule/30045/view_talk
raw-playlist
raw-mp4-playlist
encoded-files-playlist
host
archive
public
mp4
svg
png
assets
release.pdf
Big_Data_Deduplication_and_data_matching_using_Python.json
logs
Admin:
episode
episode list
cut list
raw files day
marks day
marks day
image_files
State:
---------
borked
edit
encode
push to queue
post
richard
review 1
email
review 2
make public
tweet
to-miror
conf
done
Locked:
clear this to unlock
Locked by:
user/process that locked.
Start:
initially scheduled time from master, adjusted to match reality
Duration:
length in hh:mm:ss
Name:
Video Title (shows in video search results)
Emails:
email(s) of the presenter(s)
Released:
has someone authorised pubication
Unknown
Yes
No
Normalise:
Channelcopy:
m=mono, 01=copy left to right, 10=right to left, 00=ignore.
Thumbnail:
filename.png
Description:
markdown
Andrew Rowe will present the lessons learnt and techniques used to process very large amounts of data from the ABS Census. The Australian Bureau of Statistics used Python to investigate data from the 2006 Australian Census. Python is an integral part of ABS systems to determine duplicated entries and link people in the Census to other ABS collections. You will learn about: * Handling large data. * Dealing with confidentiality. * Multiprocessing techniques. * Performance tips and tricks. * Difference between if( 1 < 2 ) and if 1 < 2.
Comment:
production notes
2013-07-07/14:06:39.dv
Apply:
14:06:39 - 14:20:19 ( 00:13:40 )
S:
14:06:39 -
E:
14:20:19
D:
00:13:40
show more...
vlc ~/Videos/veyepar/pyconau/pycon_au_2013/dv/Tasman_B/2013-07-07/14:06:39.dv :start-time=00.0 --audio-desync=0
Raw File
Cut List
14:06:39
seconds: 0.0
Wall: 14:06:39
Duration
00:13:40
14:20:19
seconds: 0.0
Wall: 14:06:39
Comments:
mp4
mp4.m3u
dv.m3u
Split:
Sequence:
:
delete
2013-07-07/14:20:20.dv
Apply:
14:20:20 - 14:44:21 ( 00:24:01 )
S:
14:20:20 -
E:
14:44:21
D:
00:24:01
show more...
vlc ~/Videos/veyepar/pyconau/pycon_au_2013/dv/Tasman_B/2013-07-07/14:20:20.dv :start-time=00.0 --audio-desync=0
Raw File
Cut List
14:20:20
seconds: 0.0
Wall: 14:20:20
Duration
00:24:01
14:44:21
seconds: 0.0
Wall: 14:20:20
Comments:
mp4
mp4.m3u
dv.m3u
Split:
Sequence:
:
delete
2013-07-07/14:44:21.dv
Apply:
14:44:21 - 14:49:37 ( 00:05:16 )
S:
14:44:21 -
E:
14:49:37
D:
00:05:16
show more...
vlc ~/Videos/veyepar/pyconau/pycon_au_2013/dv/Tasman_B/2013-07-07/14:44:21.dv :start-time=00.0 --audio-desync=0
Raw File
Cut List
14:44:21
seconds: 0.0
Wall: 14:44:21
Duration
00:05:16
14:49:37
seconds: 0.0
Wall: 14:44:21
Comments:
mp4
mp4.m3u
dv.m3u
Split:
Sequence:
:
delete
2013-07-07/14:49:38.dv
Apply:
14:49:38 - 15:12:40 ( 00:23:02 )
S:
14:49:38 -
E:
15:12:40
D:
00:23:02
show more...
vlc ~/Videos/veyepar/pyconau/pycon_au_2013/dv/Tasman_B/2013-07-07/14:49:38.dv :start-time=00.0 --audio-desync=0
Raw File
Cut List
14:49:38
seconds: 0.0
Wall: 14:49:38
Duration
00:23:02
15:12:40
seconds: 0.0
Wall: 14:49:38
Comments:
mp4
mp4.m3u
dv.m3u
Split:
Sequence:
:
delete
2013-07-07/15:12:40.dv
Apply:
15:12:40 - 15:36:20 ( 00:23:40 )
S:
15:12:40 -
E:
15:36:20
D:
00:23:40
show more...
vlc ~/Videos/veyepar/pyconau/pycon_au_2013/dv/Tasman_B/2013-07-07/15:12:40.dv :start-time=00.0 --audio-desync=0
Raw File
Cut List
15:12:40
seconds: 0.0
Wall: 15:12:40
Duration
00:23:40
15:36:20
seconds: 0.0
Wall: 15:12:40
Comments:
mp4
mp4.m3u
dv.m3u
Split:
Sequence:
:
delete
Rf filename:
root is .../show/dv/location/, example: 2013-03-13/13:13:30.dv
Sequence:
get this:
check and save to add this
2013-07-07/14:06:39.dv
2013-07-07/14:20:20.dv
2013-07-07/14:44:21.dv
2013-07-07/14:49:38.dv
2013-07-07/15:12:40.dv
Veyepar
Video Eyeball Processor and Review
