Hybrid Solution Analysis of Streaming Sensor Data with Spark & Kafka on Bluemix
Today's analytics applications must not only be capable of handling ever increasing types of data (Variety), they need to do it in near real time (Velocity). Exponentially increasing amounts of data (Volume) need to be processed to yield correct actions (Veracity). These new analytics applications must also be flexible in terms of deployment - employing a combination of cloud and on-premise resources as required to meet the needs of the business. This type of hybrid cloud, near real time analytic application requires streaming analytics capabilities, messaging systems and secure gateways to integrate cloud and on-premise systems.
In this meetup, we'll walk through how to create such an application. Simulated sensor data will be pre-processed in a node.js cloud application, transported via TCP to an on-premise Apache Kafka messaging system over a secure gateway, and monitored and analyzed with a Spark Streaming application running on Hadoop. We will explore the various technology components used in this application as well as demonstrate how they integrate in an end-to-end application flow. We will also do a deep dive into the Spark Streaming application itself.
Agenda:
- Networking, pizza, soda, snacks
- Overview presentation covering the underlying technology components
- Demo showing the various components of the application
- Discussion with Q&A
Note: Visithttps://ibm.biz/NYC_Meetupto sign up for a free Bluemix account if you don't have one!