veyepar: From Idea to Product: Customer Profiling in Apache Zeppelin with PySpark

Author(s):	Sarah Sprich
Location	Baobab
Date	oct Thu 11	Days Raw Files
Start	12:20	First Raw Start	11:49
Duration	00:45:00	Offset	0:30:15
End	13:05	Last Raw End	13:30
Chapters	00:00 0:02:40 0:24:39 0:32:39
Total cuts_time	36 min.

https://za.pycon.org/talks/61-from-idea-to-product-customer-profiling-in-apache-zeppelin-with-pyspark/
raw-playlist raw-mp4-playlist encoded-files-playlist host archive tweet mp4 svg png

assets release.pdf From_Idea_to_Product_Customer_Profiling_in_Apache_Zeppelin_with_PySpark.json
logs
Admin: episode episode list cut list raw files day marks day marks day image_files

State:

Locked:

clear this to unlock

Locked by:

user/process that locked.

Start:

initially scheduled time from master, adjusted to match reality

Duration:

length in hh:mm:ss

Name:

Video Title (shows in video search results)

Emails:

email(s) of the presenter(s)

Released:

has someone authorised pubication

Normalise:

Channelcopy:

m=mono, 01=copy left to right, 10=right to left, 00=ignore.

Thumbnail:

filename.png

Description:

markdown

Zeppelin is a web based notebook which enables interactive data analytics on big data.  Data can easily be ingested from a variety of databases and analysis can be performed in Python and Pyspark.  Visualisations can be built and displayed together with the code, using Zeppelin’s built in tool Helium, or Python specific tools such as Matplotlib and Bokeh.  The web based interface facilitates easy sharing of results, and collaboration on projects.
 
Developing in Zeppelin has changed the way we approach model development.  We are able to take a project from an idea to a product all within one tool using the following process:
<ol>
    <li>Come up with an idea. Write some notes in a Zeppelin notebook describing how we would like the idea implemented.</li>
    <li>Slowly start fleshing out the idea, with real code, until the solution is built.  This is great to demo, as the code is in bite size chunks, and visualisations can be added directly in.</li> 
    <li>Take the code into production.  It can be scheduled it to run directly in Zeppelin with a cron scheduler, or from a tool such as Nifi.  Interactive visualisations can be embedded in a web-based frontend.</li>
</ol>
 
This talk is aimed at data scientists, particularly those working with big data.  We will demonstrate how we have built a catalogue of subscriber attributes based on customer mobile usage and purchase behavior using Zeppelin and Pyspark.  These attributes can be used to profile subscribers, and are the starting point for indivisualised customer engagement.  Anyone who attends this talk will get an introduction to Zeppelin and Pyspark and an overview of what can be achieved with these tools.

Comment:

production notes