veyepar: Distributed Deep Neural Network Training using MPI on Python

Author(s):	Arpan Jain, Kawthar Shafie Khorassani
Location	Hays Cape
Date	jul Sat 27	Days Raw Files
Start	July 27, 2019, 11:15 a.m.	First Raw Start	July 19, 2019, 7:46 p.m.
Duration	0:30:0	Offset	7 days, 15:28:36
End	July 27, 2019, 11:45 a.m.	Last Raw End	July 19, 2019, 9:16 p.m.
Chapters	00:00 0:00:01 0:12:47
Total cuts_time	29 min.

https://www.pyohio.org/2019/presentations/123
raw-playlist raw-mp4-playlist encoded-files-playlist host archive tweet mp4 svg png

assets release.pdf Distributed_Deep_Neural_Network_Training_using_MPI_on_Python.json
logs
Admin: episode episode list cut list raw files day marks day marks day image_files

State:

Locked:

clear this to unlock

Locked by:

user/process that locked.

Start:

initially scheduled time from master, adjusted to match reality

Duration:

length in hh:mm:ss

Name:

Video Title (shows in video search results)

Emails:

email(s) of the presenter(s)

Released:

has someone authorised pubication

Normalise:

Channelcopy:

m=mono, 01=copy left to right, 10=right to left, 00=ignore.

Thumbnail:

filename.png

Description:

markdown

Deep learning models are a subset of machine learning models and algorithms which are designed to induce Artificial Intelligence in computers. The rise of deep learning can be attributed to the presence of large datasets and growing computational power. Deep learning models are used in face recognition, speech recognition, and many other applications. TensorFlow is a popular deep learning framework for python used to implement and train Deep Neural Networks (DNNs). Message Passing Interface (MPI) is a programming paradigm, often used in parallel applications, that allows processes to communicate with each other. Horovod provides an interface in python to couple DNN written using TensorFlow and MPI to train DNNs in less amount of time using the distributed training approach. MPI functions are optimized to provide multiple communication routines including point-to-point and collective communication. Point-to-point communication refers to a communication pattern that involves a sender process and a receiver process while collective communication involves a group of processes exchanging messages. In particular, the reduction is a collective function widely used in deep learning models to perform group operations. In this talk, we intend to demonstrate the challenges and elements to consider for DNN training using MPI in Python.

Deep Learning(DL) has attracted a lot of attention in recent years, and python has been the front runner language when it comes to the framework and implementation. Training of DL models remains a challenge as it requires a huge amount of time and computational resources. We will discuss the distributed training of the Deep Neural Network using the MPI across multiple GPUs or CPUs.

Comment:

production notes