veyepar: Scraping Your Way to a Dataset

Author(s):	Alex Zharichenko
Location	Cartoon 1
Date	jul Sat 27	Days Raw Files
Start	14:00	First Raw Start	13:32
Duration	0:45:0	Offset	0:27:21
End	14:45	Last Raw End	15:02
Chapters	00:00 0:03:13 0:33:12
Total cuts_time	41 min.

https://www.pyohio.org/2019/presentations/66
raw-playlist raw-mp4-playlist encoded-files-playlist host archive tweet mp4 svg png

assets release.pdf Scraping_Your_Way_to_a_Dataset.json
logs
Admin: episode episode list cut list raw files day marks day marks day image_files

State:

Locked:

clear this to unlock

Locked by:

user/process that locked.

Start:

initially scheduled time from master, adjusted to match reality

Duration:

length in hh:mm:ss

Name:

Video Title (shows in video search results)

Emails:

email(s) of the presenter(s)

Released:

has someone authorised pubication

Normalise:

Channelcopy:

m=mono, 01=copy left to right, 10=right to left, 00=ignore.

Thumbnail:

filename.png

Description:

markdown

It is essential to have a very large and high-quality dataset in order to perform significant analytics or to use in various machine learning tasks. For some tasks, there exists simple APIs or repositories of data to collect from. But for many other tasks like tracking prices of products, predicting stock prices, and predicting outcomes of sports games there isn't a convenient way to retrieve this information besides a webpage. Because of these circumstances, learning to scrape data from webpages and other sources allows us to create our own dataset. Additionally, scraping grants us the ability to ask better questions about data in the world.

This talk is geared towards beginner-to-intermediate Python developers that want to be able to ask and answer better questions through data. This talk will provide a guide for web scraping through two examples, and it will explain how to get the scraped data into a usable form. Throughout the talk, I will highlight some tips for improving scraper performance, minimizing the risk that a web server will stop you, and different ways to store the collected data. The first of the two examples will examine a simple case of scraping data about the lottery and the second will explore a more challenging case of scraping course information from a University.

Large datasets are vital for the majority of analytic and machine learning tasks. But what happens when the data you need isn't available in some convenient and easily obtainable form? This talk will go through the process of data scraping to create a dataset that can be then used for various analytical or machine learning tasks.

Comment:

production notes