Presenter: Michael Armbrust
Abstract: Delta Lake is an open source storage layer that brings reliability to data lakes. Delta Lake offers ACID transactions, scalable metadata handling, & unifies streaming & batch data processing. It runs on top of your existing data lake & is fully compatible with Apache Spark APIs.
In this talk, we will coverWhat data quality problems Delta helps addressHow to convert your existing application to deltaHow the Delta transaction protocol works internallyThe Delta roadmap for the next few releasesHow to get involved!
Bio: Michael Armbrust is a committer & PMC member of Apache Spark & the original creator of Spark SQL. He currently leads the team at Databricks that designed & built Structured Streaming & the Delta Lake open source project. He received his PhD from UC Berkeley in 2013, & was advised by Michael Franklin, David Patterson, & Armando Fox. His thesis focused on building systems that allow developers to rapidly build scalable interactive applications, & specifically defined the notion of scale independence. His interests broadly include distributed systems, large-scale structured storage & query optimization.