Hi
user
Admin Login:
Username:
Password:
Name:
Parallel High Performance Statistical Bootstrapping in Python
--client
enthought
--show
scipy_2012
--room Ballroom_CDE 1184 --force
Next: 11 Lightning Talks - Wednesday
show more...
Marks
Author(s):
Aakash Prasad
Location
Ballroom (CDE)
Date
jul Wed 18
Days Raw Files
Start
15:30
First Raw Start
15:04
Duration
20:00
Offset
0:25:37
End
15:50
Last Raw End
15:55
Chapters
00:00
0:10:51
Total cuts_time
12 min.
raw-playlist
raw-mp4-playlist
encoded-files-playlist
host
archive
public
tweet
mp4
svg
png
assets
release.pdf
Parallel_High_Performance_Statistical_Bootstrapping_in_Python.json
logs
Admin:
episode
episode list
cut list
raw files day
marks day
marks day
image_files
State:
---------
borked
edit
encode
push to queue
post
richard
review 1
email
review 2
make public
tweet
to-miror
conf
done
Locked:
clear this to unlock
Locked by:
user/process that locked.
Start:
initially scheduled time from master, adjusted to match reality
Duration:
length in hh:mm:ss
Name:
Video Title (shows in video search results)
Emails:
email(s) of the presenter(s)
Released:
has someone authorised pubication
Unknown
Yes
No
Normalise:
Channelcopy:
m=mono, 01=copy left to right, 10=right to left, 00=ignore.
Thumbnail:
filename.png
Description:
markdown
BLB ("Bag of Little Bootstraps") is a method to assess the quality of a statistical estimator based upon subsets of sample distributions. BLB is a variant of and solves the same class of problems as the general bootstrap. Unfortunately, the general bootstrap is a computationally demanding operation when given large data sets, and does not parallelize easily. BLB is an attractive alternative due to its its structural and computational properties which allow for much better parallelization. However, two obstacles exist to realizing this parallelism in practice. First, expressing the parallelism inherent in the algorithm requires quite different code depending on the platform (for example, multi-core with Cilk-aware compiler vs. GPU with CUDA or OpenCL vs. shared-nothing cloud computing with Spark or Hadoop). Second, even given the skeleton code for a particular platform, the specific estimator function being computed is different for each application, making it difficult to encapsulate the BLB pattern in a library. We apply the SEJITS technology (Selective Embedded Just-in-Time Specialization) to solve both problems: scientists can write applications in Python that make use of estimator functions also written in (a subset of) Python, and just-in-time code generation techniques are used to "lower" these functions to efficiency-level languages and inline them into an execution template optimized for each type of parallel platform; in this paper we focus on the multicore environment. The result is that Python applications that use BLB are source- and performance-portable, with performance comparable to hand-written code in low-level languages. We expect that code variants produced for the multicore environment can reasonably support data sets in the order of tens of gigabytes in size, and that variants for the cloud environment can support data sets in the order of terabytes in size. Preliminary results from a simple application of linear regression show a 13.6x speedup as a result of using 16 cores instead of 1 core, and this parallel performance was obtained by simply coding the linear estimator function in Python. Our runtime compiler (specializer) for BLB augments a growing family of such compilers that will ultimately allow Python applications to make use of generic computational patterns across different platforms.
Comment:
production notes
<?xml version='1.0' encoding='UTF-8'?> <ns0:entry xmlns:ns0="http://www.w3.org/2005/Atom"><ns0:category scheme="http://schemas.google.com/g/2005#kind" term="http://gdata.youtube.com/schemas/2007#video" /><ns0:category label="Education" scheme="http://gdata.youtube.com/schemas/2007/categories.cat" term="Education" /><ns0:category scheme="http://gdata.youtube.com/schemas/2007/keywords.cat" term="enthought" /><ns0:category scheme="http://gdata.youtube.com/schemas/2007/keywords.cat" term="scipy_2012" /><ns0:category scheme="http://gdata.youtube.com/schemas/2007/keywords.cat" term="HPC" /><ns0:category scheme="http://gdata.youtube.com/schemas/2007/keywords.cat" term="AakashPrasad" /><ns0:category scheme="http://gdata.youtube.com/schemas/2007/keywords.cat" term="DavidHoward" /><ns0:category scheme="http://gdata.youtube.com/schemas/2007/keywords.cat" term="ShoaibKamil" /><ns0:category scheme="http://gdata.youtube.com/schemas/2007/keywords.cat" term="ArmandoFox" /><ns0:id>http://gdata.youtube.com/feeds/api/videos/pwx5lPBjI54</ns0:id><ns0:author><ns0:name>NextDayVideo</ns0:name><ns0:uri>https://gdata.youtube.com/feeds/api/users/NextDayVideo</ns0:uri></ns0:author><ns0:content type="text">Aakash Prasad, David Howard, Shoaib Kamil, Armando Fox</ns0:content><ns0:updated>2012-08-02T22:35:58.000Z</ns0:updated><ns0:published>2012-08-02T22:35:58.000Z</ns0:published><ns1:comments xmlns:ns1="http://schemas.google.com/g/2005"><ns1:feedLink countHint="0" href="https://gdata.youtube.com/feeds/api/videos/pwx5lPBjI54/comments?client=ndv" rel="http://gdata.youtube.com/schemas/2007#comments" /></ns1:comments><ns1:group xmlns:ns1="http://search.yahoo.com/mrss/"><ns1:keywords>enthought, scipy_2012, HPC, AakashPrasad, DavidHoward, ShoaibKamil, ArmandoFox</ns1:keywords><ns1:description type="plain">Aakash Prasad, David Howard, Shoaib Kamil, Armando Fox</ns1:description><ns1:title type="plain">Parallel High Performance Statistical Bootstrapping in Python</ns1:title><ns2:duration seconds="0" xmlns:ns2="http://gdata.youtube.com/schemas/2007" /><ns1:content duration="0" expression="full" isDefault="true" medium="video" type="application/x-shockwave-flash" url="https://www.youtube.com/v/pwx5lPBjI54?version=3&f=user_uploads&c=ndv&d=Aarb2r5skm2_yNTuKEinXdAO88HsQjpE1a8d1GxQnGDm&app=youtube_gdata" ns2:format="5" xmlns:ns2="http://gdata.youtube.com/schemas/2007" /><ns1:thumbnail height="360" time="00:00:00" url="http://i.ytimg.com/vi/pwx5lPBjI54/0.jpg" width="480" /><ns1:thumbnail height="90" time="00:00:00" url="http://i.ytimg.com/vi/pwx5lPBjI54/1.jpg" width="120" /><ns1:thumbnail height="90" time="00:00:00" url="http://i.ytimg.com/vi/pwx5lPBjI54/2.jpg" width="120" /><ns1:thumbnail height="90" time="00:00:00" url="http://i.ytimg.com/vi/pwx5lPBjI54/3.jpg" width="120" /><ns1:category label="Education" scheme="http://gdata.youtube.com/schemas/2007/categories.cat">Education</ns1:category><ns1:category scheme="http://gdata.youtube.com/schemas/2007/developertags.cat">enthought</ns1:category><ns1:category scheme="http://gdata.youtube.com/schemas/2007/developertags.cat">scipy_2012</ns1:category><ns1:category scheme="http://gdata.youtube.com/schemas/2007/developertags.cat">HPC</ns1:category><ns1:category scheme="http://gdata.youtube.com/schemas/2007/developertags.cat">AakashPrasad</ns1:category><ns1:category scheme="http://gdata.youtube.com/schemas/2007/developertags.cat">DavidHoward</ns1:category><ns1:category scheme="http://gdata.youtube.com/schemas/2007/developertags.cat">ShoaibKamil</ns1:category><ns1:category scheme="http://gdata.youtube.com/schemas/2007/developertags.cat">ArmandoFox</ns1:category><ns1:player url="https://www.youtube.com/watch?v=pwx5lPBjI54&feature=youtube_gdata_player" /></ns1:group><ns0:title type="text">Parallel High Performance Statistical Bootstrapping in Python</ns0:title><ns1:control xmlns:ns1="http://purl.org/atom/app#"><ns1:draft>yes</ns1:draft><ns2:state name="processing" xmlns:ns2="http://gdata.youtube.com/schemas/2007" /></ns1:control><ns0:link href="https://www.youtube.com/watch?v=pwx5lPBjI54&feature=youtube_gdata" rel="alternate" type="text/html" /><ns0:link href="https://gdata.youtube.com/feeds/api/videos/pwx5lPBjI54/responses?client=ndv" rel="http://gdata.youtube.com/schemas/2007#video.responses" type="application/atom+xml" /><ns0:link href="https://gdata.youtube.com/feeds/api/videos/pwx5lPBjI54/ratings?client=ndv" rel="http://gdata.youtube.com/schemas/2007#video.ratings" type="application/atom+xml" /><ns0:link href="https://gdata.youtube.com/feeds/api/videos/pwx5lPBjI54/complaints?client=ndv" rel="http://gdata.youtube.com/schemas/2007#video.complaints" type="application/atom+xml" /><ns0:link href="https://gdata.youtube.com/feeds/api/videos/pwx5lPBjI54/related?client=ndv" rel="http://gdata.youtube.com/schemas/2007#video.related" type="application/atom+xml" /><ns0:link href="https://gdata.youtube.com/feeds/api/users/nextdayvideo/uploads/pwx5lPBjI54?client=ndv" rel="self" type="application/atom+xml" /><ns0:link href="https://gdata.youtube.com/feeds/api/users/nextdayvideo/uploads/pwx5lPBjI54?client=ndv" rel="edit" type="application/atom+xml" /></ns0:entry> <?xml version='1.0' encoding='UTF-8'?> <ns0:entry xmlns:ns0="http://www.w3.org/2005/Atom" xmlns:ns1="http://schemas.google.com/g/2005" xmlns:ns2="http://search.yahoo.com/mrss/" xmlns:ns3="http://gdata.youtube.com/schemas/2007" xmlns:ns4="http://purl.org/atom/app#"><ns0:category scheme="http://schemas.google.com/g/2005#kind" term="http://gdata.youtube.com/schemas/2007#video" /><ns0:category label="Education" scheme="http://gdata.youtube.com/schemas/2007/categories.cat" term="Education" /><ns0:category scheme="http://gdata.youtube.com/schemas/2007/keywords.cat" term="enthought" /><ns0:category scheme="http://gdata.youtube.com/schemas/2007/keywords.cat" term="scipy_2012" /><ns0:category scheme="http://gdata.youtube.com/schemas/2007/keywords.cat" term="HPC" /><ns0:category scheme="http://gdata.youtube.com/schemas/2007/keywords.cat" term="AakashPrasad" /><ns0:id>http://gdata.youtube.com/feeds/api/videos/vxEOYn_wedQ</ns0:id><ns0:author><ns0:name>NextDayVideo</ns0:name><ns0:uri>https://gdata.youtube.com/feeds/api/users/NextDayVideo</ns0:uri></ns0:author><ns0:content type="text">Aakash Prasad BLB ("Bag of Little Bootstraps") is a method to assess the quality of a statistical estimator based upon subsets of sample distributions. BLB is a variant of and solves the same class of problems as the general bootstrap. Unfortunately,</ns0:content><ns0:updated>2012-09-04T06:22:51.000Z</ns0:updated><ns0:published>2012-09-04T06:22:51.000Z</ns0:published><ns1:comments><ns1:feedLink countHint="0" href="https://gdata.youtube.com/feeds/api/videos/vxEOYn_wedQ/comments?client=NextDayVideo" rel="http://gdata.youtube.com/schemas/2007#comments" /></ns1:comments><ns2:group><ns2:keywords>enthought, scipy_2012, HPC, AakashPrasad</ns2:keywords><ns2:description type="plain">Aakash Prasad BLB ("Bag of Little Bootstraps") is a method to assess the quality of a statistical estimator based upon subsets of sample distributions. BLB is a variant of and solves the same class of problems as the general bootstrap. Unfortunately,</ns2:description><ns2:title type="plain">Parallel High Performance Statistical Bootstrapping in Python</ns2:title><ns3:duration seconds="0" /><ns2:content duration="0" expression="full" isDefault="true" medium="video" type="application/x-shockwave-flash" url="https://www.youtube.com/v/vxEOYn_wedQ?version=3&f=user_uploads&c=NextDayVideo&d=Aarb2r5skm2_yNTuKEinXdAO88HsQjpE1a8d1GxQnGDm&app=youtube_gdata" ns3:format="5" /><ns2:thumbnail height="360" time="00:00:00" url="http://i.ytimg.com/vi/vxEOYn_wedQ/0.jpg" width="480" /><ns2:thumbnail height="90" time="00:00:00" url="http://i.ytimg.com/vi/vxEOYn_wedQ/1.jpg" width="120" /><ns2:thumbnail height="90" time="00:00:00" url="http://i.ytimg.com/vi/vxEOYn_wedQ/2.jpg" width="120" /><ns2:thumbnail height="90" time="00:00:00" url="http://i.ytimg.com/vi/vxEOYn_wedQ/3.jpg" width="120" /><ns2:category label="Education" scheme="http://gdata.youtube.com/schemas/2007/categories.cat">Education</ns2:category><ns2:category scheme="http://gdata.youtube.com/schemas/2007/developertags.cat">enthought</ns2:category><ns2:category scheme="http://gdata.youtube.com/schemas/2007/developertags.cat">scipy_2012</ns2:category><ns2:category scheme="http://gdata.youtube.com/schemas/2007/developertags.cat">HPC</ns2:category><ns2:category scheme="http://gdata.youtube.com/schemas/2007/developertags.cat">AakashPrasad</ns2:category><ns2:player url="https://www.youtube.com/watch?v=vxEOYn_wedQ&feature=youtube_gdata_player" /></ns2:group><ns0:title type="text">Parallel High Performance Statistical Bootstrapping in Python</ns0:title><ns4:control><ns4:draft>yes</ns4:draft><ns3:state name="processing" /></ns4:control><ns0:link href="https://www.youtube.com/watch?v=vxEOYn_wedQ&feature=youtube_gdata" rel="alternate" type="text/html" /><ns0:link href="https://gdata.youtube.com/feeds/api/videos/vxEOYn_wedQ/responses?client=NextDayVideo" rel="http://gdata.youtube.com/schemas/2007#video.responses" type="application/atom+xml" /><ns0:link href="https://gdata.youtube.com/feeds/api/videos/vxEOYn_wedQ/ratings?client=NextDayVideo" rel="http://gdata.youtube.com/schemas/2007#video.ratings" type="application/atom+xml" /><ns0:link href="https://gdata.youtube.com/feeds/api/videos/vxEOYn_wedQ/complaints?client=NextDayVideo" rel="http://gdata.youtube.com/schemas/2007#video.complaints" type="application/atom+xml" /><ns0:link href="https://gdata.youtube.com/feeds/api/videos/vxEOYn_wedQ/related?client=NextDayVideo" rel="http://gdata.youtube.com/schemas/2007#video.related" type="application/atom+xml" /><ns0:link href="https://gdata.youtube.com/feeds/api/users/nextdayvideo/uploads/vxEOYn_wedQ?client=NextDayVideo" rel="self" type="application/atom+xml" /><ns0:link href="https://gdata.youtube.com/feeds/api/users/nextdayvideo/uploads/vxEOYn_wedQ?client=NextDayVideo" rel="edit" type="application/atom+xml" /></ns0:entry>
2012-07-18/13_04_23.dv
Apply:
15:04:23 - 15:30:28 ( 00:26:05 )
S:
15:04:23 -
E:
15:30:28
D:
00:26:05
show more...
vlc ~/Videos/veyepar/enthought/scipy_2012/dv/Ballroom_CDE/2012-07-18/13_04_23.dv :start-time=00.0 --audio-desync=0
Raw File
Cut List
15:04:23
seconds: 0.0
Wall: 15:04:23
Duration
00:26:05
15:30:28
seconds: 0.0
Wall: 15:04:23
Comments:
mp4
mp4.m3u
dv.m3u
Split:
Sequence:
:
delete
2012-07-18/13_30_30.dv
Apply:
15:30:30 - 15:41:21 ( 00:10:51 )
S:
15:30:30 -
E:
15:41:21
D:
00:10:51
show more...
vlc ~/Videos/veyepar/enthought/scipy_2012/dv/Ballroom_CDE/2012-07-18/13_30_30.dv :start-time=00.0 --audio-desync=0
Raw File
Cut List
15:30:30
seconds: 0.0
Wall: 15:30:30
Duration
00:10:51
15:41:21
seconds: 0.0
Wall: 15:30:30
Comments:
mp4
mp4.m3u
dv.m3u
Split:
Sequence:
:
delete
2012-07-18/13_41_21.dv
Apply:
15:41:21 - 15:42:31 ( 00:01:10 )
S:
15:41:21 -
E:
15:42:31
D:
00:01:10
show more...
vlc ~/Videos/veyepar/enthought/scipy_2012/dv/Ballroom_CDE/2012-07-18/13_41_21.dv :start-time=00.0 --audio-desync=0
Raw File
Cut List
15:41:21
seconds: 0.0
Wall: 15:41:21
Duration
00:01:10
15:42:31
seconds: 0.0
Wall: 15:41:21
Comments:
mp4
mp4.m3u
dv.m3u
Split:
Sequence:
:
delete
2012-07-18/13_42_32.dv
Apply:
15:42:32 - 15:43:51 ( 00:01:19 )
S:
15:42:32 -
E:
15:43:51
D:
00:01:19
show more...
vlc ~/Videos/veyepar/enthought/scipy_2012/dv/Ballroom_CDE/2012-07-18/13_42_32.dv :start-time=00.0 --audio-desync=0
Raw File
Cut List
15:42:32
seconds: 0.0
Wall: 15:42:32
Duration
00:01:19
15:43:51
seconds: 0.0
Wall: 15:42:32
Comments:
mp4
mp4.m3u
dv.m3u
Split:
Sequence:
:
delete
2012-07-18/13_44_21.dv
Apply:
15:44:21 - 15:55:33 ( 00:11:12 )
S:
15:44:21 -
E:
15:55:33
D:
00:11:12
show more...
vlc ~/Videos/veyepar/enthought/scipy_2012/dv/Ballroom_CDE/2012-07-18/13_44_21.dv :start-time=00.0 --audio-desync=0
Raw File
Cut List
15:44:21
seconds: 0.0
Wall: 15:44:21
Duration
00:11:12
15:55:33
seconds: 0.0
Wall: 15:44:21
Comments:
mp4
mp4.m3u
dv.m3u
Split:
Sequence:
:
delete
Rf filename:
root is .../show/dv/location/, example: 2013-03-13/13:13:30.dv
Sequence:
get this:
check and save to add this
2012-07-18/13_04_23.dv
2012-07-18/13_30_30.dv
2012-07-18/13_41_21.dv
2012-07-18/13_42_32.dv
2012-07-18/13_44_21.dv
Veyepar
Video Eyeball Processor and Review
