| |
|
EVENT DETAILS |
Agenda
12:00 pm -- 12:05 pm member join
12:05 pm -- 12:10 pm Introduction
12:10 pm -- 12:55 pm Talk + QA
1:00 pm -- Closing
Please register the event at https://www.aicamp.ai/event/eventdetails/W2021062112
We work with AICamp to host the event, AICamp provide the zoom service for us.
Topic: Delight -- free & cross-platform monitoring dashboard for Apache Spark
https://github.com/datamechanics/delight
* Delight provides CPU & Memory metrics, aligned with Spark jobs & stages information, that make it easy to find the performance bottleneck or the failure reason of a Spark pipeline.
* Delight gives you access to the Spark UI (so it runs a Spark History Server under the hood)
* Delight helps keep track of your cloud costs, & know which apps are inefficient & worth optimizing
* Delight works by installing an open-source agent on your Spark infrastructure, it works on top of any platform, commercial (Databricks, EMR, Dataproc, Glue, HDinsight, HDP/CDP) or open-source.
* Delight is free to use, & the data collected by Delight is automatically deleted after 30 days.
Speaker will go through concrete troubleshooting & performance tuning sessions with Delight on real-world data pipelines - with live demo, & iteration on code so you can see how to take action based on Delight's insights.
This should be an interesting talk for engineers who want to better understand how Spark works & how to analyze the performance of their jobs.
Speakers: Jean-Yves Stephan ( Data Mechanics)
Jean-Yves Stephan: JY is the CEO & Co-Founder of Data Mechanics, a serverless Spark platform that automatically & dynamically tunes the infrastructure parameters & Spark configurations of pipelines running on it. Prior to that, he was a software engineer at Databricks leading their Spark infrastructure team.
|
|
|
|
|
|
