veyepar: Big Data De-Duping

Hi user

Admin Login:

Name: Big Data De-Duping
--client chipy --show june2012 --room room_1 1092 --force
Prev: 4 Open Government Data Movement
Next: 0 The Python Open Data Summit Intro

show more...

Author(s):	Forest Gregg
Location	Room 1
Date	jun Thu 14	Days Raw Files
Start	18:05	First Raw Start	18:05
Duration	45:00	Offset	0:00:00
End	18:50	Last Raw End	19:09
Chapters	00:00 0:34:49 0:34:49
Total cuts_time	41 min.

raw-playlist raw-mp4-playlist encoded-files-playlist host mp4 svg png

final audio

assets release.pdf Big_Data_DeDuping.json
logs
Admin: episode episode list cut list raw files day marks day marks day image_files

State:
Locked:	clear this to unlock
Locked by:	user/process that locked.
Start:	initially scheduled time from master, adjusted to match reality
Duration:	length in hh:mm:ss
Name:	Video Title (shows in video search results)
Emails:	email(s) of the presenter(s)
Released:	has someone authorised pubication
Normalise:
Channelcopy:	m=mono, 01=copy left to right, 10=right to left, 00=ignore.
Thumbnail:	filename.png
Description:	Derek Eder of Webitects and Forest Gregg, a Ph.D. student of sociology at the U of C, will describe the Python library they are developing to deduplicate tabular data, quickly, accurately, and at a large scale. The library facilitates the matching of related records in different data sets, using a machine learning approach. They expect to have a demo to show and will explain how they expect that the library will be used. markdown
Comment:	<?xml version='1.0' encoding='UTF-8'?> <ns0:entry xmlns:ns0="http://www.w3.org/2005/Atom" xmlns:ns1="http://schemas.google.com/g/2005" xmlns:ns2="http://search.yahoo.com/mrss/" xmlns:ns3="http://gdata.youtube.com/schemas/2007" xmlns:ns4="http://purl.org/atom/app#"><ns0:category scheme="http://schemas.google.com/g/2005#kind" term="http://gdata.youtube.com/schemas/2007#video" /><ns0:category label="Education" scheme="http://gdata.youtube.com/schemas/2007/categories.cat" term="Education" /><ns0:category scheme="http://gdata.youtube.com/schemas/2007/keywords.cat" term="chipy" /><ns0:category scheme="http://gdata.youtube.com/schemas/2007/keywords.cat" term="june2012" /><ns0:category scheme="http://gdata.youtube.com/schemas/2007/keywords.cat" term="ForestGregg" /><ns0:id>http://gdata.youtube.com/feeds/api/videos/iG7Mlvdrdzk</ns0:id><ns0:author><ns0:name>cfkarsten</ns0:name><ns0:uri>https://gdata.youtube.com/feeds/api/users/cfkarsten</ns0:uri></ns0:author><ns0:content type="text">Forest Gregg Derek Eder of Webitects and Forest Gregg, a Ph.D. student of sociology at the U of C, will describe the Python library they are developing to deduplicate tabular data, quickly, accurately, and at a large scale. The library facilitates th</ns0:content><ns0:updated>2012-06-15T22:31:14.000Z</ns0:updated><ns0:published>2012-06-15T22:31:14.000Z</ns0:published><ns1:comments><ns1:feedLink countHint="0" href="https://gdata.youtube.com/feeds/api/videos/iG7Mlvdrdzk/comments?client=cfkarsten" rel="http://gdata.youtube.com/schemas/2007#comments" /></ns1:comments><ns2:group><ns2:keywords>chipy, june2012, ForestGregg</ns2:keywords><ns2:description type="plain">Forest Gregg Derek Eder of Webitects and Forest Gregg, a Ph.D. student of sociology at the U of C, will describe the Python library they are developing to deduplicate tabular data, quickly, accurately, and at a large scale. The library facilitates th</ns2:description><ns2:title type="plain">Big Data De-Duping</ns2:title><ns3:duration seconds="0" /><ns2:content duration="0" expression="full" isDefault="true" medium="video" type="application/x-shockwave-flash" url="https://www.youtube.com/v/iG7Mlvdrdzk?version=3&f=user_uploads&c=cfkarsten&d=Aarb2r5skm2_yNTuKEinXdAO88HsQjpE1a8d1GxQnGDm&app=youtube_gdata" ns3:format="5" /><ns2:thumbnail height="360" time="00:00:00" url="http://i.ytimg.com/vi/iG7Mlvdrdzk/0.jpg" width="480" /><ns2:thumbnail height="90" time="00:00:00" url="http://i.ytimg.com/vi/iG7Mlvdrdzk/1.jpg" width="120" /><ns2:thumbnail height="90" time="00:00:00" url="http://i.ytimg.com/vi/iG7Mlvdrdzk/2.jpg" width="120" /><ns2:thumbnail height="90" time="00:00:00" url="http://i.ytimg.com/vi/iG7Mlvdrdzk/3.jpg" width="120" /><ns2:category label="Education" scheme="http://gdata.youtube.com/schemas/2007/categories.cat">Education</ns2:category><ns2:category scheme="http://gdata.youtube.com/schemas/2007/developertags.cat">chipy</ns2:category><ns2:category scheme="http://gdata.youtube.com/schemas/2007/developertags.cat">june2012</ns2:category><ns2:category scheme="http://gdata.youtube.com/schemas/2007/developertags.cat">ForestGregg</ns2:category><ns2:player url="https://www.youtube.com/watch?v=iG7Mlvdrdzk&feature=youtube_gdata_player" /></ns2:group><ns0:title type="text">Big Data De-Duping</ns0:title><ns4:control><ns4:draft>yes</ns4:draft><ns3:state name="processing" /></ns4:control><ns0:link href="https://www.youtube.com/watch?v=iG7Mlvdrdzk&feature=youtube_gdata" rel="alternate" type="text/html" /><ns0:link href="https://gdata.youtube.com/feeds/api/videos/iG7Mlvdrdzk/responses?client=cfkarsten" rel="http://gdata.youtube.com/schemas/2007#video.responses" type="application/atom+xml" /><ns0:link href="https://gdata.youtube.com/feeds/api/videos/iG7Mlvdrdzk/ratings?client=cfkarsten" rel="http://gdata.youtube.com/schemas/2007#video.ratings" type="application/atom+xml" /><ns0:link href="https://gdata.youtube.com/feeds/api/videos/iG7Mlvdrdzk/complaints?client=cfkarsten" rel="http://gdata.youtube.com/schemas/2007#video.complaints" type="application/atom+xml" /><ns0:link href="https://gdata.youtube.com/feeds/api/videos/iG7Mlvdrdzk/related?client=cfkarsten" rel="http://gdata.youtube.com/schemas/2007#video.related" type="application/atom+xml" /><ns0:link href="https://gdata.youtube.com/feeds/api/users/cfkarsten/uploads/iG7Mlvdrdzk?client=cfkarsten" rel="self" type="application/atom+xml" /><ns0:link href="https://gdata.youtube.com/feeds/api/users/cfkarsten/uploads/iG7Mlvdrdzk?client=cfkarsten" rel="edit" type="application/atom+xml" /></ns0:entry> <?xml version='1.0' encoding='UTF-8'?> <ns0:entry xmlns:ns0="http://www.w3.org/2005/Atom" xmlns:ns1="http://schemas.google.com/g/2005" xmlns:ns2="http://search.yahoo.com/mrss/" xmlns:ns3="http://gdata.youtube.com/schemas/2007" xmlns:ns4="http://purl.org/atom/app#"><ns0:category scheme="http://schemas.google.com/g/2005#kind" term="http://gdata.youtube.com/schemas/2007#video" /><ns0:category label="Education" scheme="http://gdata.youtube.com/schemas/2007/categories.cat" term="Education" /><ns0:category scheme="http://gdata.youtube.com/schemas/2007/keywords.cat" term="chipy" /><ns0:category scheme="http://gdata.youtube.com/schemas/2007/keywords.cat" term="june2012" /><ns0:category scheme="http://gdata.youtube.com/schemas/2007/keywords.cat" term="ForestGregg" /><ns0:id>http://gdata.youtube.com/feeds/api/videos/7Lwlo42j6aA</ns0:id><ns0:author><ns0:name>cfkarsten</ns0:name><ns0:uri>https://gdata.youtube.com/feeds/api/users/cfkarsten</ns0:uri></ns0:author><ns0:content type="text">Forest Gregg Derek Eder of Webitects and Forest Gregg, a Ph.D. student of sociology at the U of C, will describe the Python library they are developing to deduplicate tabular data, quickly, accurately, and at a large scale. The library facilitates th</ns0:content><ns0:updated>2012-06-16T00:39:34.000Z</ns0:updated><ns0:published>2012-06-16T00:39:34.000Z</ns0:published><ns1:comments><ns1:feedLink countHint="0" href="https://gdata.youtube.com/feeds/api/videos/7Lwlo42j6aA/comments?client=cfkarsten" rel="http://gdata.youtube.com/schemas/2007#comments" /></ns1:comments><ns2:group><ns2:keywords>chipy, june2012, ForestGregg</ns2:keywords><ns2:description type="plain">Forest Gregg Derek Eder of Webitects and Forest Gregg, a Ph.D. student of sociology at the U of C, will describe the Python library they are developing to deduplicate tabular data, quickly, accurately, and at a large scale. The library facilitates th</ns2:description><ns2:title type="plain">Big Data De-Duping</ns2:title><ns3:duration seconds="0" /><ns2:content duration="0" expression="full" isDefault="true" medium="video" type="application/x-shockwave-flash" url="https://www.youtube.com/v/7Lwlo42j6aA?version=3&f=user_uploads&c=cfkarsten&d=Aarb2r5skm2_yNTuKEinXdAO88HsQjpE1a8d1GxQnGDm&app=youtube_gdata" ns3:format="5" /><ns2:thumbnail height="360" time="00:00:00" url="http://i.ytimg.com/vi/7Lwlo42j6aA/0.jpg" width="480" /><ns2:thumbnail height="90" time="00:00:00" url="http://i.ytimg.com/vi/7Lwlo42j6aA/1.jpg" width="120" /><ns2:thumbnail height="90" time="00:00:00" url="http://i.ytimg.com/vi/7Lwlo42j6aA/2.jpg" width="120" /><ns2:thumbnail height="90" time="00:00:00" url="http://i.ytimg.com/vi/7Lwlo42j6aA/3.jpg" width="120" /><ns2:category label="Education" scheme="http://gdata.youtube.com/schemas/2007/categories.cat">Education</ns2:category><ns2:category scheme="http://gdata.youtube.com/schemas/2007/developertags.cat">chipy</ns2:category><ns2:category scheme="http://gdata.youtube.com/schemas/2007/developertags.cat">june2012</ns2:category><ns2:category scheme="http://gdata.youtube.com/schemas/2007/developertags.cat">ForestGregg</ns2:category><ns2:player url="https://www.youtube.com/watch?v=7Lwlo42j6aA&feature=youtube_gdata_player" /></ns2:group><ns0:title type="text">Big Data De-Duping</ns0:title><ns4:control><ns4:draft>yes</ns4:draft><ns3:state name="processing" /></ns4:control><ns0:link href="https://www.youtube.com/watch?v=7Lwlo42j6aA&feature=youtube_gdata" rel="alternate" type="text/html" /><ns0:link href="https://gdata.youtube.com/feeds/api/videos/7Lwlo42j6aA/responses?client=cfkarsten" rel="http://gdata.youtube.com/schemas/2007#video.responses" type="application/atom+xml" /><ns0:link href="https://gdata.youtube.com/feeds/api/videos/7Lwlo42j6aA/ratings?client=cfkarsten" rel="http://gdata.youtube.com/schemas/2007#video.ratings" type="application/atom+xml" /><ns0:link href="https://gdata.youtube.com/feeds/api/videos/7Lwlo42j6aA/complaints?client=cfkarsten" rel="http://gdata.youtube.com/schemas/2007#video.complaints" type="application/atom+xml" /><ns0:link href="https://gdata.youtube.com/feeds/api/videos/7Lwlo42j6aA/related?client=cfkarsten" rel="http://gdata.youtube.com/schemas/2007#video.related" type="application/atom+xml" /><ns0:link href="https://gdata.youtube.com/feeds/api/users/cfkarsten/uploads/7Lwlo42j6aA?client=cfkarsten" rel="self" type="application/atom+xml" /><ns0:link href="https://gdata.youtube.com/feeds/api/users/cfkarsten/uploads/7Lwlo42j6aA?client=cfkarsten" rel="edit" type="application/atom+xml" /></ns0:entry> production notes

2012-06-14/18_05_56-1.dv Apply: 18:05:56 - 18:40:45 ( 00:34:49 ) S: 18:05:56 - E: 18:40:45 D: 00:34:49 show more...

vlc ~/Videos/veyepar/chipy/june2012/dv/room_1/2012-06-14/18_05_56-1.dv :start-time=00.0 --audio-desync=0

	Raw File	Cut List
	18:05:56		seconds: 0.0 Wall: 18:05:56
Duration	00:34:49
	18:40:45		seconds: 0.0 Wall: 18:05:56
Comments:
mp4 mp4.m3u	dv.m3u	Split: Sequence:: delete

audio viz

2012-06-14/18_05_56.dv Apply: 18:05:56 - 18:05:56 ( 00:00:00 ) S: 18:05:56 - E: 18:05:56 D: 00:00:00 show more...

vlc ~/Videos/veyepar/chipy/june2012/dv/room_1/2012-06-14/18_05_56.dv :start-time=00.0 --audio-desync=0

	Raw File	Cut List
	18:05:56		seconds: 0.0 Wall: 18:05:56
Duration	00:00:00
	18:05:56		seconds: 0.0 Wall: 18:05:56
Comments:
mp4 mp4.m3u	dv.m3u	Split: Sequence:: delete

audio viz

2012-06-14/18_40_46.dv Apply: 18:40:46 - 18:47:45 ( 00:06:59 ) S: 18:40:46 - E: 18:47:45 D: 00:06:59 show more...

vlc ~/Videos/veyepar/chipy/june2012/dv/room_1/2012-06-14/18_40_46.dv :start-time=00.0 --audio-desync=0

	Raw File	Cut List
	18:40:46		seconds: 0.0 Wall: 18:40:46
Duration	00:06:59
	18:47:45		seconds: 0.0 Wall: 18:40:46
Comments:
mp4 mp4.m3u	dv.m3u	Split: Sequence:: delete

audio viz

2012-06-14/18_47_46.dv Apply: 18:47:46 - 18:47:46 ( 00:00:00 ) S: 18:47:46 - E: 18:47:46 D: 00:00:00 show more...

vlc ~/Videos/veyepar/chipy/june2012/dv/room_1/2012-06-14/18_47_46.dv :start-time=00.0 --audio-desync=0

	Raw File	Cut List
	18:47:46		seconds: 0.0 Wall: 18:47:46
Duration	00:00:00
	18:47:46		seconds: 0.0 Wall: 18:47:46
Comments:
mp4 mp4.m3u	dv.m3u	Split: Sequence:: delete

audio viz

2012-06-14/18_47_46-1.dv Apply: 18:47:46 - 19:09:59 ( 00:22:13 ) S: 18:47:46 - E: 19:09:59 D: 00:22:13 show more...

vlc ~/Videos/veyepar/chipy/june2012/dv/room_1/2012-06-14/18_47_46-1.dv :start-time=00.0 --audio-desync=0

	Raw File	Cut List
	18:47:46		seconds: 0.0 Wall: 18:47:46
Duration	00:22:13
	19:09:59		seconds: 0.0 Wall: 18:47:46
Comments:
mp4 mp4.m3u	dv.m3u	Split: Sequence:: delete

audio viz

Rf filename:	root is .../show/dv/location/, example: 2013-03-13/13:13:30.dv
Sequence:
get this:	check and save to add this

2012-06-14/18_05_56-1.dv
audio viz

audio viz

2012-06-14/18_05_56.dv
audio viz

audio viz

2012-06-14/18_40_46.dv
audio viz

audio viz

2012-06-14/18_47_46-1.dv
audio viz

audio viz

2012-06-14/18_47_46.dv
audio viz

audio viz

Veyepar Video Eyeball Processor and Review