Talk 1: Tensorflow on Apache Hadoop YARN

Tensorflow is one of the most popular open source projects
for machine learning & deep learning, which can handle enterprise use cases like image recognition, video analytics, audio translation, etc. However, training deep learning model was very expensive which requires lots of GPU resources. Also, a real-life distributed Tensorflow application needs a bunch of services such as workers, parameter servers, TensorBoard, etc. work together. Those services need to be carefully configured to make them can talk to each other.

To make distributed TF applications can be easily launched,
managed, monitored by YARN, we introduced YARN service assembly along with other improvements such as GPU support, container-DNS support, scheduling improvements, etc. These improvements make distributed TF applications can run on YARN as simple as run it
locally, which can let TF developers focus on deep learning algorithms instead of worrying about underlying infrastructure. Also, YARN can better manage a shared cluster which runs TF & other services/batch jobs with these improvements.

During this session, we will take a closer look at these improvements, & we will do a demo of running a distributed TF assembly which consists of workers, parameter servers, TensorBoard & prediction servers on YARN.

Speaker: Wangda Tan

Wangda Tan is Product Management Committee
(PMC) member of Apache Hadoop & Staff Software Engineer at Hortonworks. His major working field is Hadoop YARN resource scheduler, participated features like node labeling, resource preemption, container resizing etc. Before join Hortonworks, he was working at Pivotal, working on integration OpenMPI/GraphLab with Hadoop YARN. Before that, he was working at Alibaba, participated creating a large scale machine learning, matrix & statistics computation platform using Map-Reduce & MPI.

Talk2 : Light Talk
TBD

Agenda
6 -- 6:40 pm check-in & networking/light dinner
6:40 -- 6:50 pm Introduction & Announcement
6:50 -- 7:50 pm Main Talk + QA
7:55 -- 8:10 pm Lighting Talk + QA
8:15 -- 8:30 pm Networking
8:30 pm -- -closing