Hi
user
Admin Login:
Username:
Password:
Name:
Solving the import problem: Scalable Dynamic Loading Network File Systems
--client
enthought
--show
scipy_2012
--room 106 1172 --force
Next: 11 Implicit Multicore Parallelism using CnC-Python
show more...
Marks
Author(s):
Aron Ahmadia, Jed Brown, William Scullin
Location
106
Date
jul Wed 18
Days Raw Files
Start
11:05
First Raw Start
10:52
Duration
20:00
Offset
0:12:16
End
11:25
Last Raw End
11:31
Chapters
00:00
Total cuts_time
23 min.
raw-playlist
raw-mp4-playlist
encoded-files-playlist
host
archive
public
tweet
mp4
svg
png
assets
release.pdf
Solving_the_import_problem_Scalable_Dynamic_Loading_Network_File_Systems.json
logs
Admin:
episode
episode list
cut list
raw files day
marks day
marks day
image_files
State:
---------
borked
edit
encode
push to queue
post
richard
review 1
email
review 2
make public
tweet
to-miror
conf
done
Locked:
clear this to unlock
Locked by:
user/process that locked.
Start:
initially scheduled time from master, adjusted to match reality
Duration:
length in hh:mm:ss
Name:
Video Title (shows in video search results)
Emails:
email(s) of the presenter(s)
Released:
has someone authorised pubication
Unknown
Yes
No
Normalise:
Channelcopy:
m=mono, 01=copy left to right, 10=right to left, 00=ignore.
Thumbnail:
filename.png
Description:
markdown
The most common programming paradigm for scientific computing, SPMD (Single Program Multiple Data), catastrophically interacts with the loading strategies of dynamically linked executables and network-attached file systems on even moderately sized high performance computing clusters. This difficulty is further exacerbated by "function-shipped" I/O on modern supercomputer compute nodes, preventing the deployment of simple solutions. In this talk, we introduce a two-component solution: collfs, a set of low-level MPI-collective file operations that can selectively shadow file system access in a library, and walla, a set of Python import hooks for seamlessly enabling parallel dynamic loading scalable to tens of thousands of cores.
Comment:
production notes
<?xml version='1.0' encoding='UTF-8'?> <ns0:entry xmlns:ns0="http://www.w3.org/2005/Atom"><ns0:category scheme="http://schemas.google.com/g/2005#kind" term="http://gdata.youtube.com/schemas/2007#video" /><ns0:category label="Education" scheme="http://gdata.youtube.com/schemas/2007/categories.cat" term="Education" /><ns0:category scheme="http://gdata.youtube.com/schemas/2007/keywords.cat" term="enthought" /><ns0:category scheme="http://gdata.youtube.com/schemas/2007/keywords.cat" term="scipy_2012" /><ns0:category scheme="http://gdata.youtube.com/schemas/2007/keywords.cat" term="HPC" /><ns0:category scheme="http://gdata.youtube.com/schemas/2007/keywords.cat" term="AronAhmadia" /><ns0:category scheme="http://gdata.youtube.com/schemas/2007/keywords.cat" term="JedBrown" /><ns0:category scheme="http://gdata.youtube.com/schemas/2007/keywords.cat" term="WilliamScullin" /><ns0:id>http://gdata.youtube.com/feeds/api/videos/BpuykTOy4a0</ns0:id><ns0:author><ns0:name>NextDayVideo</ns0:name><ns0:uri>https://gdata.youtube.com/feeds/api/users/NextDayVideo</ns0:uri></ns0:author><ns0:content type="text">Aron Ahmadia, Jed Brown, William Scullin</ns0:content><ns0:updated>2012-08-02T22:06:31.000Z</ns0:updated><ns0:published>2012-08-02T22:06:31.000Z</ns0:published><ns1:comments xmlns:ns1="http://schemas.google.com/g/2005"><ns1:feedLink countHint="0" href="https://gdata.youtube.com/feeds/api/videos/BpuykTOy4a0/comments?client=ndv" rel="http://gdata.youtube.com/schemas/2007#comments" /></ns1:comments><ns1:group xmlns:ns1="http://search.yahoo.com/mrss/"><ns1:keywords>enthought, scipy_2012, HPC, AronAhmadia, JedBrown, WilliamScullin</ns1:keywords><ns1:description type="plain">Aron Ahmadia, Jed Brown, William Scullin</ns1:description><ns1:title type="plain">Solving the import problem: Scalable Dynamic Loading Network File Systems</ns1:title><ns2:duration seconds="0" xmlns:ns2="http://gdata.youtube.com/schemas/2007" /><ns1:content duration="0" expression="full" isDefault="true" medium="video" type="application/x-shockwave-flash" url="https://www.youtube.com/v/BpuykTOy4a0?version=3&f=user_uploads&c=ndv&d=Aarb2r5skm2_yNTuKEinXdAO88HsQjpE1a8d1GxQnGDm&app=youtube_gdata" ns2:format="5" xmlns:ns2="http://gdata.youtube.com/schemas/2007" /><ns1:thumbnail height="360" time="00:00:00" url="http://i.ytimg.com/vi/BpuykTOy4a0/0.jpg" width="480" /><ns1:thumbnail height="90" time="00:00:00" url="http://i.ytimg.com/vi/BpuykTOy4a0/1.jpg" width="120" /><ns1:thumbnail height="90" time="00:00:00" url="http://i.ytimg.com/vi/BpuykTOy4a0/2.jpg" width="120" /><ns1:thumbnail height="90" time="00:00:00" url="http://i.ytimg.com/vi/BpuykTOy4a0/3.jpg" width="120" /><ns1:category label="Education" scheme="http://gdata.youtube.com/schemas/2007/categories.cat">Education</ns1:category><ns1:category scheme="http://gdata.youtube.com/schemas/2007/developertags.cat">enthought</ns1:category><ns1:category scheme="http://gdata.youtube.com/schemas/2007/developertags.cat">scipy_2012</ns1:category><ns1:category scheme="http://gdata.youtube.com/schemas/2007/developertags.cat">HPC</ns1:category><ns1:category scheme="http://gdata.youtube.com/schemas/2007/developertags.cat">AronAhmadia</ns1:category><ns1:category scheme="http://gdata.youtube.com/schemas/2007/developertags.cat">JedBrown</ns1:category><ns1:category scheme="http://gdata.youtube.com/schemas/2007/developertags.cat">WilliamScullin</ns1:category><ns1:player url="https://www.youtube.com/watch?v=BpuykTOy4a0&feature=youtube_gdata_player" /></ns1:group><ns0:title type="text">Solving the import problem: Scalable Dynamic Loading Network File Systems</ns0:title><ns1:control xmlns:ns1="http://purl.org/atom/app#"><ns1:draft>yes</ns1:draft><ns2:state name="processing" xmlns:ns2="http://gdata.youtube.com/schemas/2007" /></ns1:control><ns0:link href="https://www.youtube.com/watch?v=BpuykTOy4a0&feature=youtube_gdata" rel="alternate" type="text/html" /><ns0:link href="https://gdata.youtube.com/feeds/api/videos/BpuykTOy4a0/responses?client=ndv" rel="http://gdata.youtube.com/schemas/2007#video.responses" type="application/atom+xml" /><ns0:link href="https://gdata.youtube.com/feeds/api/videos/BpuykTOy4a0/ratings?client=ndv" rel="http://gdata.youtube.com/schemas/2007#video.ratings" type="application/atom+xml" /><ns0:link href="https://gdata.youtube.com/feeds/api/videos/BpuykTOy4a0/complaints?client=ndv" rel="http://gdata.youtube.com/schemas/2007#video.complaints" type="application/atom+xml" /><ns0:link href="https://gdata.youtube.com/feeds/api/videos/BpuykTOy4a0/related?client=ndv" rel="http://gdata.youtube.com/schemas/2007#video.related" type="application/atom+xml" /><ns0:link href="https://gdata.youtube.com/feeds/api/users/nextdayvideo/uploads/BpuykTOy4a0?client=ndv" rel="self" type="application/atom+xml" /><ns0:link href="https://gdata.youtube.com/feeds/api/users/nextdayvideo/uploads/BpuykTOy4a0?client=ndv" rel="edit" type="application/atom+xml" /></ns0:entry>
2012-07-18/08_52_44.dv
Apply:
10:52:44 - 11:07:22 ( 00:14:38 )
S:
10:52:44 -
E:
11:07:22
D:
00:14:38
show more...
vlc ~/Videos/veyepar/enthought/scipy_2012/dv/106/2012-07-18/08_52_44.dv :start-time=00.0 --audio-desync=0
Raw File
Cut List
10:52:44
seconds: 0.0
Wall: 10:52:44
Duration
00:14:38
11:07:22
seconds: 0.0
Wall: 10:52:44
Comments:
mp4
mp4.m3u
dv.m3u
Split:
Sequence:
:
delete
2012-07-18/09_07_23.dv
Apply:
11:07:23 - 11:07:28 ( 00:00:05 )
S:
11:07:23 -
E:
11:07:28
D:
00:00:05
show more...
vlc ~/Videos/veyepar/enthought/scipy_2012/dv/106/2012-07-18/09_07_23.dv :start-time=00.0 --audio-desync=0
Raw File
Cut List
11:07:23
seconds: 0.0
Wall: 11:07:23
Duration
00:00:05
11:07:28
seconds: 0.0
Wall: 11:07:23
Comments:
mp4
mp4.m3u
dv.m3u
Split:
Sequence:
:
delete
2012-07-18/09_07_29.dv
Apply:
11:07:29 - 11:31:18 ( 00:23:49 )
S:
11:07:29 -
E:
11:31:18
D:
00:23:49
show more...
vlc ~/Videos/veyepar/enthought/scipy_2012/dv/106/2012-07-18/09_07_29.dv :start-time=00.0 --audio-desync=0
Raw File
Cut List
11:07:29
seconds: 0.0
Wall: 11:07:29
Duration
00:23:49
11:31:18
seconds: 0.0
Wall: 11:07:29
Comments:
mp4
mp4.m3u
dv.m3u
Split:
Sequence:
:
delete
Rf filename:
root is .../show/dv/location/, example: 2013-03-13/13:13:30.dv
Sequence:
get this:
check and save to add this
2012-07-18/08_52_44.dv
2012-07-18/09_07_23.dv
2012-07-18/09_07_29.dv
Veyepar
Video Eyeball Processor and Review