Cloud storage products, such as S3, have many advantages including data security mechanisms and high reliability. However, S3 is much slower than HDFS and direct attached storage. In this presentation, we dive into some of the improvementsQubolehas made to Hadoop and Hive to speed up communication with S3. These changes have resulted in dramatic performance improvements (8x in specific query processing steps detailed in this post). And most importantly they have significantly improved the experience of our users. Ashish Dubey and Jason Huang, Solutions Architects withQubolewill walk you through the optimizations and provide a short technical demonstration.

About the speakers:

Jason Huang

Jason is a Senior Solutions Architect atQubole, where he works with prospects and customers to perform massive ad hoc queries in the public cloud. Prior to joiningQubole, Jason served as a technical architect for enterprise solutions spanning centralized logging, business intelligence, data analytics, private cloud infrastructure, distributed systems and computational grids within financial services and Fortune 500 organizations. He was previously a Senior Solutions Consultant with TIBCO and DataSynapse. Jason holds an BA in Computer Science from Brown University.

Ashish Dubey

Ashish is a Senior Solutions Architect atQubolewith more than 10 years of experience. Strong experience with design/development of high volume Big Data projects, integration with BI tools. Prior toQubole, Ashish was a Solutions Architect for Lunexa, a design engineer at Microsoft, and holds a Bachelors degree in Computer Science from the Motilal Nehru National Institute Of Technology.