Hi
user
Admin Login:
Username:
Password:
Name:
Scale Data Science by Pandas API on Spark
--client
sfpy
--show
pybay23
--room bungalo_west 15329 --force
Next: 10 Beyond Conventional: Embracing Python & LLMs for Quality Assurance
show more...
Marks
Author(s):
Xinrong Meng
Location
Bungalo West
Date
oct Sun 08
Days Raw Files
Start
17:00
First Raw Start
16:25
Duration
00:25:00
Offset
0:34:11
End
17:25
Last Raw End
17:55
Chapters
00:00
0:25:10
Total cuts_time
26 min.
https://pybay.com/speakers/#sz-speaker-584fc381-f0db-4da5-ae69-d2092d26bb3a
raw-playlist
raw-mp4-playlist
encoded-files-playlist
host
mp4
svg
png
assets
release.pdf
Scale_Data_Science_by_Pandas_API_on_Spark.json
logs
Admin:
episode
episode list
cut list
raw files day
marks day
marks day
image_files
State:
---------
borked
edit
encode
push to queue
post
richard
review 1
email
review 2
make public
tweet
to-miror
conf
done
Locked:
clear this to unlock
Locked by:
user/process that locked.
Start:
initially scheduled time from master, adjusted to match reality
Duration:
length in hh:mm:ss
Name:
Video Title (shows in video search results)
Emails:
email(s) of the presenter(s)
Released:
has someone authorised pubication
Unknown
Yes
No
Normalise:
Channelcopy:
m=mono, 01=copy left to right, 10=right to left, 00=ignore.
Thumbnail:
filename.png
Description:
markdown
As Python has become the go-to language for data science, pandas has quickly evolved into a standard library in the field. However, one key drawback of pandas is its inability to linearly scale with increasing data volumes, primarily due to its reliance on single-machine processing. Pandas API on Spark addresses this issue, empowering users to handle vast datasets by leveraging Apache Spark while preserving the pandas APIs. In this talk, I will introduce the Pandas API on Spark, explain how it enables the scaling of data science workloads, and explore the reasons behind its highly optimized performance. By the end of the session, you will have the knowledge to scale your existing data science workloads seamlessly using this powerful tool.
Comment:
production notes
2023-10-08/16_25_49.ts
Apply:
16:25:49 - 16:39:51 ( 00:14:02 )
S:
16:25:49 -
E:
16:55:48
D:
00:29:59
(
End:
842.0)
show more...
vlc ~/Videos/veyepar/sfpy/pybay23/dv/bungalo_west/2023-10-08/16_25_49.ts :start-time=00.0 --audio-desync=0
Raw File
Cut List
16:25:49
seconds: 0.0
Wall: 16:25:49
Duration
00:29:59
16:55:48
seconds: 842.0
Wall: 16:39:51
Comments:
mp4
mp4.m3u
dv.m3u
Split:
Sequence:
:
delete
2023-10-08/16_25_49.ts
Apply:
16:39:51 - 16:40:06 ( 00:00:15 )
S:
16:25:49 -
E:
16:55:48
D:
00:29:59
(
Start:
842.0) (
End:
857.0)
show more...
vlc ~/Videos/veyepar/sfpy/pybay23/dv/bungalo_west/2023-10-08/16_25_49.ts :start-time=0842.0 --audio-desync=0
Raw File
Cut List
16:25:49
seconds: 842.0
Wall: 16:39:51
Duration
00:29:59
16:55:48
seconds: 857.0
Wall: 16:40:06
Comments:
mp4
mp4.m3u
dv.m3u
Split:
Sequence:
:
delete
2023-10-08/16_25_49.ts
Apply:
16:40:06 - 16:42:15 ( 00:02:09 )
S:
16:25:49 -
E:
16:55:48
D:
00:29:59
(
Start:
857.0) (
End:
986.0)
show more...
vlc ~/Videos/veyepar/sfpy/pybay23/dv/bungalo_west/2023-10-08/16_25_49.ts :start-time=0857.0 --audio-desync=0
Raw File
Cut List
16:25:49
seconds: 857.0
Wall: 16:40:06
Duration
00:29:59
16:55:48
seconds: 986.0
Wall: 16:42:15
Comments:
mp4
mp4.m3u
dv.m3u
Split:
Sequence:
:
delete
2023-10-08/16_25_49.ts
Apply:
16:42:15 - 16:52:24 ( 00:10:09 )
S:
16:25:49 -
E:
16:55:48
D:
00:29:59
(
Start:
986.0) (
End:
1595.0)
show more...
vlc ~/Videos/veyepar/sfpy/pybay23/dv/bungalo_west/2023-10-08/16_25_49.ts :start-time=0986.0 --audio-desync=0
Raw File
Cut List
16:25:49
seconds: 986.0
Wall: 16:42:15
Duration
00:29:59
16:55:48
seconds: 1595.0
Wall: 16:52:24
Comments:
mp4
mp4.m3u
dv.m3u
Split:
Sequence:
:
delete
2023-10-08/16_25_49.ts
Apply:
16:52:24 - 16:52:25 ( 00:00:01 )
S:
16:25:49 -
E:
16:55:48
D:
00:29:59
(
Start:
1595.0) (
End:
1596.0)
show more...
vlc ~/Videos/veyepar/sfpy/pybay23/dv/bungalo_west/2023-10-08/16_25_49.ts :start-time=01595.0 --audio-desync=0
Raw File
Cut List
16:25:49
seconds: 1595.0
Wall: 16:52:24
Duration
00:29:59
16:55:48
seconds: 1596.0
Wall: 16:52:25
Comments:
mp4
mp4.m3u
dv.m3u
Split:
Sequence:
:
delete
2023-10-08/16_25_49.ts
Apply:
16:52:25 - 16:52:26 ( 00:00:01 )
S:
16:25:49 -
E:
16:55:48
D:
00:29:59
(
Start:
1596.0) (
End:
1597.0)
show more...
vlc ~/Videos/veyepar/sfpy/pybay23/dv/bungalo_west/2023-10-08/16_25_49.ts :start-time=01596.0 --audio-desync=0
Raw File
Cut List
16:25:49
seconds: 1596.0
Wall: 16:52:25
Duration
00:29:59
16:55:48
seconds: 1597.0
Wall: 16:52:26
Comments:
mp4
mp4.m3u
dv.m3u
Split:
Sequence:
:
delete
2023-10-08/16_25_49.ts
Apply:
16:52:26 - 16:52:28 ( 00:00:02 )
S:
16:25:49 -
E:
16:55:48
D:
00:29:59
(
Start:
1597.0) (
End:
1599.0)
show more...
vlc ~/Videos/veyepar/sfpy/pybay23/dv/bungalo_west/2023-10-08/16_25_49.ts :start-time=01597.0 --audio-desync=0
Raw File
Cut List
16:25:49
seconds: 1597.0
Wall: 16:52:26
Duration
00:29:59
16:55:48
seconds: 1599.0
Wall: 16:52:28
Comments:
mp4
mp4.m3u
dv.m3u
Split:
Sequence:
:
delete
2023-10-08/16_25_49.ts
Apply:
16:52:28 - 16:55:48 ( 00:03:20 )
S:
16:25:49 -
E:
16:55:48
D:
00:29:59
(
Start:
1599.0)
show more...
vlc ~/Videos/veyepar/sfpy/pybay23/dv/bungalo_west/2023-10-08/16_25_49.ts :start-time=01599.0 --audio-desync=0
Raw File
Cut List
16:25:49
seconds: 1599.0
Wall: 16:52:28
Duration
00:29:59
16:55:48
seconds: 0.0
Wall: 16:25:49
Comments:
mp4
mp4.m3u
dv.m3u
Split:
Sequence:
:
delete
2023-10-08/16_55_49.ts
Apply:
16:55:49 - 16:59:07 ( 00:03:18 )
S:
16:55:49 -
E:
17:25:49
D:
00:30:00
(
End:
198.0)
show more...
vlc ~/Videos/veyepar/sfpy/pybay23/dv/bungalo_west/2023-10-08/16_55_49.ts :start-time=00.0 --audio-desync=0
Raw File
Cut List
16:55:49
seconds: 0.0
Wall: 16:55:49
Duration
00:30:00
17:25:49
seconds: 198.0
Wall: 16:59:07
Comments:
mp4
mp4.m3u
dv.m3u
Split:
Sequence:
:
delete
2023-10-08/16_55_49.ts
Apply:
16:59:07 - 16:59:42 ( 00:00:35 )
S:
16:55:49 -
E:
17:25:49
D:
00:30:00
(
Start:
198.0) (
End:
233.0)
show more...
vlc ~/Videos/veyepar/sfpy/pybay23/dv/bungalo_west/2023-10-08/16_55_49.ts :start-time=0198.0 --audio-desync=0
Raw File
Cut List
16:55:49
seconds: 198.0
Wall: 16:59:07
Duration
00:30:00
17:25:49
seconds: 233.0
Wall: 16:59:42
Comments:
mp4
mp4.m3u
dv.m3u
Split:
Sequence:
:
delete
2023-10-08/16_55_49.ts
Apply:
17:00:39 - 17:25:49 ( 00:25:10 )
S:
16:55:49 -
E:
17:25:49
D:
00:30:00
(
Start:
290)
show more...
vlc ~/Videos/veyepar/sfpy/pybay23/dv/bungalo_west/2023-10-08/16_55_49.ts :start-time=0290.0 --audio-desync=0
Raw File
Cut List
16:55:49
seconds: 290.0
Wall: 17:00:39
Duration
00:30:00
17:25:49
seconds: 0.0
Wall: 16:55:49
Comments:
mp4
mp4.m3u
dv.m3u
Split:
Sequence:
:
delete
2023-10-08/17_25_49.ts
Apply:
17:25:49 - 17:27:09 ( 00:01:20 )
S:
17:25:49 -
E:
17:55:48
D:
00:29:59
(
End:
80.0)
show more...
vlc ~/Videos/veyepar/sfpy/pybay23/dv/bungalo_west/2023-10-08/17_25_49.ts :start-time=00.0 --audio-desync=0
Raw File
Cut List
17:25:49
seconds: 0.0
Wall: 17:25:49
Duration
00:29:59
17:55:48
seconds: 80.0
Wall: 17:27:09
Comments:
mp4
mp4.m3u
dv.m3u
Split:
Sequence:
:
delete
2023-10-08/17_25_49.ts
Apply:
17:27:09 - 17:55:48 ( 00:28:39 )
S:
17:25:49 -
E:
17:55:48
D:
00:29:59
(
Start:
80.0)
show more...
vlc ~/Videos/veyepar/sfpy/pybay23/dv/bungalo_west/2023-10-08/17_25_49.ts :start-time=080.0 --audio-desync=0
Raw File
Cut List
17:25:49
seconds: 80.0
Wall: 17:27:09
Duration
00:29:59
17:55:48
seconds: 0.0
Wall: 17:25:49
Comments:
mp4
mp4.m3u
dv.m3u
Split:
Sequence:
:
delete
Rf filename:
root is .../show/dv/location/, example: 2013-03-13/13:13:30.dv
Sequence:
get this:
check and save to add this
2023-10-08/16_25_49.ts
2023-10-08/16_55_49.ts
2023-10-08/17_25_49.ts
Veyepar
Video Eyeball Processor and Review
