| |
|
| |
| DETAILS |
|
Join fellow airflow enthusiasts & leaders at Samba TV's Office for an evening of engaging presentations, delicious food & drinks, & exclusive new Airflow t-shirts!
From Samba's K8s setup to Astronomer's self-aware pipelines, real Airflow lessons from real Airflow teams.
PRESENTATIONS
Samba Airflow: The Good, The Bad, & The Ugly
Speaker: Piotr Kostrzeski, Software Engineer at Samba TV
Running Apache Airflow on Kubernetes isn't always smooth sailing. In this talk, Piotr Kostrzenski walks through Samba's Airflow deployment, from initial setup to production challenges. You'll hear what went well, what broke spectacularly, & how the team adapted their infrastructure to support large-scale data workflows. It's a candid, behind-the-scenes look at the realities of orchestrating Airflow in a containerized world.
How the Airflow Company Uses Airflow
Speaker: Steven Hillion, SVP of Data & AI at Astronomer
Astronomer's data team recently underwent a major transformation in how we use Apache Airflow. In response to several operational challenges, we re-architected our workflows to improve reliability, scalability, & observability. This shift involved adopting dataset scheduling & breaking down our pipelines into micro-pipelines to reduce failure rates & boost overall stability. We also implemented robust observability tools to manage complex dependencies & gain full end-to-end visibility into our pipelines. To streamline onboarding & support growth, we standardized the use of Task Groups across our workflows. With Airflow now managing itself more effectively, we're able to refocus on delivering value through data rather than wrestling with infrastructure. As a testament to these improvements, we'll be sharing some of our favorite stats from the terabyte of data we process daily-offering a unique look into how data teams around the world are leveraging Airflow.
Dynamic DAGs in Airflow
Speaker: Clayton Cole, Senior Data Engineer at Samba TV
In this talk, Clayton Cole will explore how to support & scale Dynamic DAGs in Apache Airflow. As data teams grow & workflows become more complex, the need for flexibility & automation in DAG generation becomes critical. Clayton will walk through key patterns, best practices, & lessons learned from building dynamic pipelines that adapt to changing data, configurations, & operational needs. Whether you're new to Dynamic DAGs or looking to level up your implementation, this session will provide practical insights & real-world examples to help you make the most of Airflow's dynamic capabilities.
AGENDA
5:30-6PM: Arrivals, eat, drink, & network
6-7:40PM: Presentations
7:40-8:30PM: Networking
|
|
|
|
|
|
|
|
