| |
|
| |
| DETAILS |
|
The Elastic Silicon Valley User Group & Bay Area Apache Kafka Meetup are partnering for a joint meetup on Tuesday, April 7th. We'll have presentations from followed by food, refreshments, & networking.
Interested in presenting a lightning talk? If you have you ever considered presenting, but think 30 or 45 minutes is too long to start with, this is a great opportunity to try it out. Interested? Please email us at meetups@elastic.co.
Date & time:
Tuesday, April 7th from 5:30 - 7:30 pm PDT
Location:
Plug & Play Tech Center - Silicon Valley Room
440 N Wolfe Rd, Sunnyvale, CA 94085
Arrival Instructions:
Attendees will need to check in at the Plug & Play lobby upon arrival. We'll provide the guest list to the venue 24 hours in advance for badge creation.
Agenda:
5:30 pm Doors open
5:45 pm: One Does Not Simply Query a Stream - Viktor Gamov, Principal Developer Advocate, Confluent
6:15: Building a Meltano Target for Elasticsearch: Automating SPACE Metrics Dashboard - DT Mirizzi, Principal Software Engineer at Palo Alto Networks
6:45 pm: Talk # 3 - Steve Leung - Sr. Solutions Architect @ Elastic
7:30 pm: Event ends
Talk Abstracts:
One Does Not Simply Query a Stream - Viktor Gamov, Principal Developer Advocate, Confluent
Streaming data with Apache Kafka has become the backbone of modern applications. While streams are ideal for continuous data flow, they lack built-in querying capabilities. Unlike databases with indexed lookups, Kafka's append-only logs are designed for high-throughput processing-not for on-demand queries. This necessitates additional infrastructure to query streaming data effectively.Traditional approaches replicate stream data into external stores: relational databases like PostgreSQL for operational queries, object storage like S3 accessed via Flink, Spark, or Trino for analytics, & Elasticsearch for full-text search & log analytics. Each serves a purpose-but they also introduce silos, schema mismatches, freshness issues, & complex ETL pipelines that increase system fragility.In this session, we'll explore solutions that aim to unify operational, analytical, & search workloads across real-time data.
We'll demonstrate stream processing with Kafka Streams, Apache Flink, & SQL engines; real-time analytics with Apache Pinot ; search capabilities with Elasticsearch; & modern lakehouse approaches using Apache Iceberg with Tableflow to represent Kafka topics as queryable tables. While there's no one-size-fits-all solution, understanding the tools & trade-offs will help you design more robust & flexible architectures.
|
|
|
|
|
|
|
|
