Dear friends,

This time we will hostKarthik Ramasamy,engineering manager and technical lead for Real Time Analytics at Twitter.

Abstract:

Storm has long served as the main platform for real-time analytics at Twitter. However, as the scale of data being processed in real- time at Twitter has increased, along with an increase in the diversity and the number of use cases, many limitations of Storm have become apparent. We need a system that scales better, has better debug-ability, has better performance, and is easier to manage all while working in a shared cluster infrastructure. We considered various alternatives to meet these needs, and in the end concluded that we needed to build a new real-time stream data processing system. This talk will present the design and implementation of the new system, called Heron. Heron is now the de facto stream data processing engine inside Twitter, and we share our experiences from running Heron in production.

Speaker Bio

Karthik is the engineering manager and technical lead for Real Time Analytics at Twitter. He has two decades of experience working in parallel databases, big data infrastructure and networking. He cofounded Locomatix, a company that specializes in real time streaming processing on Hadoop and Cassandra using SQL that was acquired by Twitter. Before Locomatix, he had a brief stint with Greenplum where he worked on parallel query scheduling. Greenplum was eventually acquired by EMC for more than $300M. Prior to Greenplum, Karthik was at Juniper Networks where he designed and delivered platforms, protocols, databases and high availability solutions for network routers that are widely deployed in the Internet. Before joining Juniper at University of Wisconsin, he worked extensively in parallel database systems, query processing, scale out technologies, storage engine and online analytical systems. Several of these research were spun as a company later acquired by Teradata.

He is the author of several publications, patents and one of the best selling book "Network Routing: Algorithms, Protocols and Architectures." He has a Ph.D. in Computer Science from UW Madison with a focus on databases.

Agenda:

6 :30 pm: Guest arrival & networking.

7: 00 pm: Twitter Heron: Stream Processing at Scale

8.30pm: event ends.

Strata + Hadoop World, happening September 29 - October 1, 2015 in New York, is where cutting-edge science and new business fundamentals intersect. It's a deep-immersion event where data scientists, analysts, and executives explore the latest in emerging techniques and technologies by dissecting case studies, developing new skills through in-depth tutorials, and sharing best practices.

Register before it's gone using our group discount code and link below.

Save 20% on any pass with discount code UGNYCSUG20.

Check out the impressive agenda and speaker lineup. Strata conference is the most interesting Big Data event in New York.