Hi
user
Admin Login:
Username:
Password:
Name:
Challenges when Scaling: Continued adventures in Swift's sharding
--client
lca
--show
lca2017
--room tasman_a 12158 --force
Next: 12 Open Compute Project down under
show more...
Marks
Author(s):
Matthew Oliver
Location
Tasman Hall A
Date
jan Tue 17
Days Raw Files
Start
22:40
First Raw Start
22:18
Duration
0:45:0
Offset
0:21:31
End
23:25
Last Raw End
23:48
Chapters
00:00
0:07:38
0:37:37
Total cuts_time
39 min.
http://linux.conf.au/schedule/presentation/36/
raw-playlist
raw-mp4-playlist
encoded-files-playlist
host
archive
tweet
mp4
svg
png
assets
release.pdf
Challenges_when_Scaling_Continued_adventures_in_Swifts_sharding.json
logs
Admin:
episode
episode list
cut list
raw files day
marks day
marks day
image_files
State:
---------
borked
edit
encode
push to queue
post
richard
review 1
email
review 2
make public
tweet
to-miror
conf
done
Locked:
clear this to unlock
Locked by:
user/process that locked.
Start:
initially scheduled time from master, adjusted to match reality
Duration:
length in hh:mm:ss
Name:
Video Title (shows in video search results)
Emails:
email(s) of the presenter(s)
Released:
has someone authorised pubication
Unknown
Yes
No
Normalise:
Channelcopy:
m=mono, 01=copy left to right, 10=right to left, 00=ignore.
Thumbnail:
filename.png
Description:
markdown
Background: Swift is an open-source, highly available, distributed, eventually-consistent object storage system. As the project has matured, so has the need to store more and more data. Some sites have moved to using expensive solid state drives (SSDs) for storage, but there has to be a better way we can solve this in software, right? This presentation continues our Swift sharding adventure from where we left it last year - but now addressing a new set of challenges we've come across in our proof of concept container sharding implementation. During testing of even larger data sets we've come across a number of limitations of the underlying SQLite database, meaning we've had to develop work arounds so that we are able to shard even larger containers. Come and see how we performed large scale testing, how we narrowed down where the limitations were, how we developed our solutions to these problems, all the while trying to maintain backwards comparability.
Comment:
production notes
2017-01-18/15_18_29.ts
Apply:
22:18:29 - 22:40:50 ( 00:22:21 )
S:
22:18:29 -
E:
22:48:28
D:
00:29:59
(
End:
1341.0)
show more...
vlc ~/Videos/veyepar/lca/lca2017/dv/tasman_a/2017-01-18/15_18_29.ts :start-time=00.0 --audio-desync=0
Raw File
Cut List
22:18:29
seconds: 0.0
Wall: 22:18:29
Duration
00:29:59
22:48:28
seconds: 1341.0
Wall: 22:40:50
Comments:
mp4
mp4.m3u
dv.m3u
Split:
Sequence:
:
delete
2017-01-18/15_18_29.ts
Apply:
22:40:50 - 22:48:28 ( 00:07:38 )
S:
22:18:29 -
E:
22:48:28
D:
00:29:59
(
Start:
1341.0)
show more...
vlc ~/Videos/veyepar/lca/lca2017/dv/tasman_a/2017-01-18/15_18_29.ts :start-time=01341.0 --audio-desync=0
Raw File
Cut List
22:18:29
seconds: 1341.0
Wall: 22:40:50
Duration
00:29:59
22:48:28
seconds: 0.0
Wall: 22:18:29
Comments:
mp4
mp4.m3u
dv.m3u
Split:
Sequence:
:
delete
2017-01-18/15_48_29.ts
Apply:
22:48:29 - 23:18:28 ( 00:29:59 )
S:
22:48:29 -
E:
23:18:28
D:
00:29:59
show more...
vlc ~/Videos/veyepar/lca/lca2017/dv/tasman_a/2017-01-18/15_48_29.ts :start-time=00.0 --audio-desync=0
Raw File
Cut List
22:48:29
seconds: 0.0
Wall: 22:48:29
Duration
00:29:59
23:18:28
seconds: 0.0
Wall: 22:48:29
Comments:
mp4
mp4.m3u
dv.m3u
Split:
Sequence:
:
delete
2017-01-18/16_18_29.ts
Apply:
23:18:29 - 23:20:23 ( 00:01:54 )
S:
23:18:29 -
E:
23:48:29
D:
00:30:00
(
End:
114.0)
show more...
vlc ~/Videos/veyepar/lca/lca2017/dv/tasman_a/2017-01-18/16_18_29.ts :start-time=00.0 --audio-desync=0
Raw File
Cut List
23:18:29
seconds: 0.0
Wall: 23:18:29
Duration
00:30:00
23:48:29
seconds: 114.0
Wall: 23:20:23
Comments:
mp4
mp4.m3u
dv.m3u
Split:
Sequence:
:
delete
2017-01-18/16_18_29.ts
Apply:
23:20:23 - 23:35:01 ( 00:14:38 )
S:
23:18:29 -
E:
23:48:29
D:
00:30:00
(
Start:
114.0) (
End:
992.0)
show more...
vlc ~/Videos/veyepar/lca/lca2017/dv/tasman_a/2017-01-18/16_18_29.ts :start-time=0114.0 --audio-desync=0
Raw File
Cut List
23:18:29
seconds: 114.0
Wall: 23:20:23
Duration
00:30:00
23:48:29
seconds: 992.0
Wall: 23:35:01
Comments:
mp4
mp4.m3u
dv.m3u
Split:
Sequence:
:
delete
2017-01-18/16_18_29.ts
Apply:
23:35:01 - 23:48:29 ( 00:13:28 )
S:
23:18:29 -
E:
23:48:29
D:
00:30:00
(
Start:
992.0)
show more...
vlc ~/Videos/veyepar/lca/lca2017/dv/tasman_a/2017-01-18/16_18_29.ts :start-time=0992.0 --audio-desync=0
Raw File
Cut List
23:18:29
seconds: 992.0
Wall: 23:35:01
Duration
00:30:00
23:48:29
seconds: 0.0
Wall: 23:18:29
Comments:
mp4
mp4.m3u
dv.m3u
Split:
Sequence:
:
delete
Rf filename:
root is .../show/dv/location/, example: 2013-03-13/13:13:30.dv
Sequence:
get this:
check and save to add this
2017-01-18/15_18_29.ts
2017-01-18/15_48_29.ts
2017-01-18/16_18_29.ts
Veyepar
Video Eyeball Processor and Review