NYC Tech Events - GarysGuide | The #1 Resource for NYC Tech

LOCATION

EVENT DETAILS

"Demystifying data engineering, with Nathan Marz" is part of Capital One Labs' "C1 Speaker Series".Previous speakers include Peter Thiel, co-founder of PayPal and Palantir. Steve Wozniak, co-founder of Apple, spoke at Capital One in 2014.
<DIV>
We are excited to host our first speaker event in NYC with our friends at AppNexus!
<DIV></DIV>
</DIV>
Join Nathan Marz (Apache Storm, Cascalog, Backtype/Twitter) and David Drummond (Insight Data Science) as they demystify data engineers.Nathan and David will talk about what data engineering really means, what their day to day looks like, strategies for identifying quality data engineers, as well as best practices for building a great team.

<HR>
 
Demystifying data engineering - Nathan Marz
A huge part of any application is answering questions about your dataset, such as who are Alice's followers, what is Bob's bank account balance, or what are related products to the one Charlie is viewing. Questions vary in their performance constraints - a question asked by a consumer facing application typically needs to respond in milliseconds, while offline analysis by data scientists may only require answers within a few hours.
It is the job of data engineers to build the infrastructure to answer these questions within their respective time constraints. The infrastructure used by a data engineer includes databases, queue systems, batch processors, stream processors, and the occasional custom server to glue things together. Along the way, they must figure out how to make it robust, manage the inevitable tradeoffs, and accomplish everything in a cost-effective manner.
I will demystify the role of "data engineer" by talking about my own experience working as one for 7 years, both for startups and as the founder of a core infrastructure team within Twitter. To achieve my goals I've had to make use of a plethora of open source tools as well as build many of my own (e.g. Apache Storm, Cascalog, ElephantDB). Building large-scale systems is never a cakewalk, and I'll share many of the battle stories I've faced along the way.
 
Finding Data Engineers: A Project-Based Approach - David Drummond
Finding quality data engineers can be a challenging task - there's a huge demand to build scalable data infrastructures and a limited supply of applicants who know the most effective techniques and tools to do so. Even with candidates that seem to have the relevant skills, the conventional interview process rarely provides enough signal to ensure that they can contribute on day one.
At the Insight Fellows Program, we help recent grads and experienced software engineers transition into roles as data scientists and engineers. We use a project-based approach that allows our Fellows to learn the latest open-source technology and helps hiring managers make informed decisions. I'll discuss the best practices for building a data team and lessons learned from helping over 200 Fellows join dozens of top tech companies in Silicon Valley and New York.
 
<HR>
 
About Nathan Marz
<IMG TITLE="Nathan Marz" SRC="https://pbs.twimg.com/profile_images/3650717510/ea74dddf70b18004268b679750c6e3e2.jpeg" ALT="Nathan Marz" WIDTH="256" HEIGHT="256">
Nathan is the creator of the open source projects Apache Storm and Cascalog which are relied upon by many of the world's largest companies. He was the lead engineer of BackType through BackType's acquisition by Twitter. At Twitter, he created the streaming compute team, a core infrastructure team within the company. He left Twitter in 2013 to work on a new startup. He is the author of the book "Big Data: Principles and best practices of scalable realtime data systems" and writes a blog at <A HREF="http://nathanmarz.com/" REL="nofollow">http://nathanmarz.com</A>.
 
About David Drummond
<IMG SRC="https://cdn.evbuc.com/eventlogos/115547123/ddrumpic.jpeg" ALT="David Drummond" WIDTH="256">

David transitioned to data engineering after completing his PhD in theoretical physics, researching fault-tolerant architectures for quantum computing. As a Program Director at Insight Data Engineering, he helps others learn the latest distributed, open source technologiesfor taming large data sets, and has helped dozens of Fellows transition to data engineering at top tech companies.

<HR>
 
<TABLE BORDER="0" FRAME="void">
<TBODY>
<TR>
<TD></TD>
<TD></TD>
</TR>
<TR>
<TD><IMG STYLE="display: block; margin-left: auto; margin-right: auto;" TITLE="Capital One Labs" SRC="https://cdn.evbuc.com/eventlogos/115547123/c1labsmark.png" ALT="Capital One Labs" WIDTH="100"></TD>
<TD>
Capital One Labs is the digital innovation group within Capital One that is re-imagining the way that people interact with their money. We are a team of product managers, engineers, designers, data scientists, and design thinking strategists working on creating products, capabilities, and experiences that deliver on customers' needs.
Learn more at <A HREF="http://www.capitalonelabs.com" TARGET="_blank" REL="nofollow">http://www.capitalonelabs.com</A>
</TD>
</TR>
<TR>
<TD></TD>
<TD></TD>
</TR>
<TR>
<TD><IMG STYLE="font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif; font-size: 13px; line-height: 1.6em; display: block; margin-left: auto; margin-right: auto;" TITLE="AppNexus" SRC="https://pbs.twimg.com/profile_images/2549744469/cqzqvf0tdkzh2d00v58k.jpeg" ALT="AppNexus" WIDTH="128" HEIGHT="128"></TD>
<TD>
AppNexus is today's most powerful, open and customizable advertising technology platform. Advertising's largest and most innovative companies build their businesses on AppNexus.
Learn more at <A HREF="http://www.appnexus.com" TARGET="_blank" REL="nofollow">http://www.appnexus.com</A>
</TD>
</TR>
</TBODY>
</TABLE>
<DIV> </DIV>
<DIV> </DIV>