veyepar: Scale Data Science by Pandas API on Spark

Author(s):	Xinrong Meng
Location	Bungalo West
Date	oct Sun 08	Days Raw Files
Start	17:00	First Raw Start	16:25
Duration	00:25:00	Offset	0:34:11
End	17:25	Last Raw End	17:55
Chapters	00:00 0:25:10
Total cuts_time	26 min.

https://pybay.com/speakers/#sz-speaker-584fc381-f0db-4da5-ae69-d2092d26bb3a
raw-playlist raw-mp4-playlist encoded-files-playlist host mp4 svg png

assets release.pdf Scale_Data_Science_by_Pandas_API_on_Spark.json
logs
Admin: episode episode list cut list raw files day marks day marks day image_files

State:
Locked:	clear this to unlock
Locked by:	user/process that locked.
Start:	initially scheduled time from master, adjusted to match reality
Duration:	length in hh:mm:ss
Name:	Video Title (shows in video search results)
Emails:	email(s) of the presenter(s)
Released:	has someone authorised pubication
Normalise:
Channelcopy:	m=mono, 01=copy left to right, 10=right to left, 00=ignore.
Thumbnail:	filename.png
Description:	As Python has become the go-to language for data science, pandas has quickly evolved into a standard library in the field. However, one key drawback of pandas is its inability to linearly scale with increasing data volumes, primarily due to its reliance on single-machine processing. Pandas API on Spark addresses this issue, empowering users to handle vast datasets by leveraging Apache Spark while preserving the pandas APIs. In this talk, I will introduce the Pandas API on Spark, explain how it enables the scaling of data science workloads, and explore the reasons behind its highly optimized performance. By the end of the session, you will have the knowledge to scale your existing data science workloads seamlessly using this powerful tool. markdown
Comment:	production notes

2023-10-08/16_25_49.ts Apply: 16:25:49 - 16:39:51 ( 00:14:02 ) S: 16:25:49 - E: 16:55:48 D: 00:29:59 (End: 842.0) show more...

vlc ~/Videos/veyepar/sfpy/pybay23/dv/bungalo_west/2023-10-08/16_25_49.ts :start-time=00.0 --audio-desync=0

	Raw File	Cut List
	16:25:49		seconds: 0.0 Wall: 16:25:49
Duration	00:29:59
	16:55:48		seconds: 842.0 Wall: 16:39:51
Comments:
mp4 mp4.m3u	dv.m3u	Split: Sequence:: delete

2023-10-08/16_25_49.ts Apply: 16:39:51 - 16:40:06 ( 00:00:15 ) S: 16:25:49 - E: 16:55:48 D: 00:29:59 (Start: 842.0) (End: 857.0) show more...

vlc ~/Videos/veyepar/sfpy/pybay23/dv/bungalo_west/2023-10-08/16_25_49.ts :start-time=0842.0 --audio-desync=0

	Raw File	Cut List
	16:25:49		seconds: 842.0 Wall: 16:39:51
Duration	00:29:59
	16:55:48		seconds: 857.0 Wall: 16:40:06
Comments:
mp4 mp4.m3u	dv.m3u	Split: Sequence:: delete

2023-10-08/16_25_49.ts Apply: 16:40:06 - 16:42:15 ( 00:02:09 ) S: 16:25:49 - E: 16:55:48 D: 00:29:59 (Start: 857.0) (End: 986.0) show more...

vlc ~/Videos/veyepar/sfpy/pybay23/dv/bungalo_west/2023-10-08/16_25_49.ts :start-time=0857.0 --audio-desync=0

	Raw File	Cut List
	16:25:49		seconds: 857.0 Wall: 16:40:06
Duration	00:29:59
	16:55:48		seconds: 986.0 Wall: 16:42:15
Comments:
mp4 mp4.m3u	dv.m3u	Split: Sequence:: delete

2023-10-08/16_25_49.ts Apply: 16:42:15 - 16:52:24 ( 00:10:09 ) S: 16:25:49 - E: 16:55:48 D: 00:29:59 (Start: 986.0) (End: 1595.0) show more...

vlc ~/Videos/veyepar/sfpy/pybay23/dv/bungalo_west/2023-10-08/16_25_49.ts :start-time=0986.0 --audio-desync=0

	Raw File	Cut List
	16:25:49		seconds: 986.0 Wall: 16:42:15
Duration	00:29:59
	16:55:48		seconds: 1595.0 Wall: 16:52:24
Comments:
mp4 mp4.m3u	dv.m3u	Split: Sequence:: delete

2023-10-08/16_25_49.ts Apply: 16:52:24 - 16:52:25 ( 00:00:01 ) S: 16:25:49 - E: 16:55:48 D: 00:29:59 (Start: 1595.0) (End: 1596.0) show more...

vlc ~/Videos/veyepar/sfpy/pybay23/dv/bungalo_west/2023-10-08/16_25_49.ts :start-time=01595.0 --audio-desync=0

	Raw File	Cut List
	16:25:49		seconds: 1595.0 Wall: 16:52:24
Duration	00:29:59
	16:55:48		seconds: 1596.0 Wall: 16:52:25
Comments:
mp4 mp4.m3u	dv.m3u	Split: Sequence:: delete

2023-10-08/16_25_49.ts Apply: 16:52:25 - 16:52:26 ( 00:00:01 ) S: 16:25:49 - E: 16:55:48 D: 00:29:59 (Start: 1596.0) (End: 1597.0) show more...

vlc ~/Videos/veyepar/sfpy/pybay23/dv/bungalo_west/2023-10-08/16_25_49.ts :start-time=01596.0 --audio-desync=0

	Raw File	Cut List
	16:25:49		seconds: 1596.0 Wall: 16:52:25
Duration	00:29:59
	16:55:48		seconds: 1597.0 Wall: 16:52:26
Comments:
mp4 mp4.m3u	dv.m3u	Split: Sequence:: delete

2023-10-08/16_25_49.ts Apply: 16:52:26 - 16:52:28 ( 00:00:02 ) S: 16:25:49 - E: 16:55:48 D: 00:29:59 (Start: 1597.0) (End: 1599.0) show more...

vlc ~/Videos/veyepar/sfpy/pybay23/dv/bungalo_west/2023-10-08/16_25_49.ts :start-time=01597.0 --audio-desync=0

	Raw File	Cut List
	16:25:49		seconds: 1597.0 Wall: 16:52:26
Duration	00:29:59
	16:55:48		seconds: 1599.0 Wall: 16:52:28
Comments:
mp4 mp4.m3u	dv.m3u	Split: Sequence:: delete

2023-10-08/16_25_49.ts Apply: 16:52:28 - 16:55:48 ( 00:03:20 ) S: 16:25:49 - E: 16:55:48 D: 00:29:59 (Start: 1599.0) show more...

vlc ~/Videos/veyepar/sfpy/pybay23/dv/bungalo_west/2023-10-08/16_25_49.ts :start-time=01599.0 --audio-desync=0

	Raw File	Cut List
	16:25:49		seconds: 1599.0 Wall: 16:52:28
Duration	00:29:59
	16:55:48		seconds: 0.0 Wall: 16:25:49
Comments:
mp4 mp4.m3u	dv.m3u	Split: Sequence:: delete

2023-10-08/16_55_49.ts Apply: 16:55:49 - 16:59:07 ( 00:03:18 ) S: 16:55:49 - E: 17:25:49 D: 00:30:00 (End: 198.0) show more...

vlc ~/Videos/veyepar/sfpy/pybay23/dv/bungalo_west/2023-10-08/16_55_49.ts :start-time=00.0 --audio-desync=0

	Raw File	Cut List
	16:55:49		seconds: 0.0 Wall: 16:55:49
Duration	00:30:00
	17:25:49		seconds: 198.0 Wall: 16:59:07
Comments:
mp4 mp4.m3u	dv.m3u	Split: Sequence:: delete

2023-10-08/16_55_49.ts Apply: 16:59:07 - 16:59:42 ( 00:00:35 ) S: 16:55:49 - E: 17:25:49 D: 00:30:00 (Start: 198.0) (End: 233.0) show more...

vlc ~/Videos/veyepar/sfpy/pybay23/dv/bungalo_west/2023-10-08/16_55_49.ts :start-time=0198.0 --audio-desync=0

	Raw File	Cut List
	16:55:49		seconds: 198.0 Wall: 16:59:07
Duration	00:30:00
	17:25:49		seconds: 233.0 Wall: 16:59:42
Comments:
mp4 mp4.m3u	dv.m3u	Split: Sequence:: delete

2023-10-08/16_55_49.ts Apply: 17:00:39 - 17:25:49 ( 00:25:10 ) S: 16:55:49 - E: 17:25:49 D: 00:30:00 (Start: 290) show more...

vlc ~/Videos/veyepar/sfpy/pybay23/dv/bungalo_west/2023-10-08/16_55_49.ts :start-time=0290.0 --audio-desync=0

	Raw File	Cut List
	16:55:49		seconds: 290.0 Wall: 17:00:39
Duration	00:30:00
	17:25:49		seconds: 0.0 Wall: 16:55:49
Comments:
mp4 mp4.m3u	dv.m3u	Split: Sequence:: delete

2023-10-08/17_25_49.ts Apply: 17:25:49 - 17:27:09 ( 00:01:20 ) S: 17:25:49 - E: 17:55:48 D: 00:29:59 (End: 80.0) show more...

vlc ~/Videos/veyepar/sfpy/pybay23/dv/bungalo_west/2023-10-08/17_25_49.ts :start-time=00.0 --audio-desync=0

	Raw File	Cut List
	17:25:49		seconds: 0.0 Wall: 17:25:49
Duration	00:29:59
	17:55:48		seconds: 80.0 Wall: 17:27:09
Comments:
mp4 mp4.m3u	dv.m3u	Split: Sequence:: delete

2023-10-08/17_25_49.ts Apply: 17:27:09 - 17:55:48 ( 00:28:39 ) S: 17:25:49 - E: 17:55:48 D: 00:29:59 (Start: 80.0) show more...

vlc ~/Videos/veyepar/sfpy/pybay23/dv/bungalo_west/2023-10-08/17_25_49.ts :start-time=080.0 --audio-desync=0

	Raw File	Cut List
	17:25:49		seconds: 80.0 Wall: 17:27:09
Duration	00:29:59
	17:55:48		seconds: 0.0 Wall: 17:25:49
Comments:
mp4 mp4.m3u	dv.m3u	Split: Sequence:: delete

Rf filename:	root is .../show/dv/location/, example: 2013-03-13/13:13:30.dv
Sequence:
get this:	check and save to add this