Presenter: Michael Armbrust

Abstract: Delta Lake is an open source storage layer that brings reliability to data lakes. Delta Lake offers ACID transactions, scalable metadata handling, & unifies streaming & batch data processing. It runs on top of your existing data lake & is fully compatible with Apache Spark APIs.

In this talk, we will cover
What data quality problems Delta helps address
How to convert your existing application to delta
How the Delta transaction protocol works internally
The Delta roadmap for the next few releases
How to get involved!

Bio: Michael Armbrust is a committer & PMC member of Apache Spark & the original creator of Spark SQL. He currently leads the team at Databricks that designed & built Structured Streaming & the Delta Lake open source project. He received his PhD from UC Berkeley in 2013, & was advised by Michael Franklin, David Patterson, & Armando Fox. His thesis focused on building systems that allow developers to rapidly build scalable interactive applications, & specifically defined the notion of scale independence. His interests broadly include distributed systems, large-scale structured storage & query optimization.