Events  Deals  Jobs  NFT NYC 2024 
    Sign in  
 
 
With Neville Li (Software Enggr, Spotify) & Lynn Root (SRE, Spotify).
Tue, Feb 25, 2020 @ 06:00 PM   FREE   Spotify, 150 Greenwich St, 62nd Fl
 
     
 
 
              

      
 
Sign up for our awesome New York
Tech Events weekly email newsletter.
   
EVENT DETAILS
NOTE: This is an event co-posted with New York Apache Beam Meetup (https://www.meetup.com/New-York-Apache-Beam/events/268153356/)

4 WTC building security requires photo ID for visitors. Please RSVP with your real name to ensure a smooth process check in process.

Food & drinks sponsored by Spotify

========================================

Talk: Scio - Data Processing at Spotify

Bio:
Neville is a data infrastructure engineer at Spotify & the creator of Scio. In the past few years he has been driving the adoption of Scala & new tools for data processing & machine learning, including Scalding, Spark, Storm & Parquet. Before that he worked on search quality at Yahoo! & old school distributed systems like MPI.

Abstract:
Scio is an open-source Scala API for Apache Beam & Google Cloud Dataflow. Its created by Spotify to process petabytes of data in both batch & streaming mode, & is adopted by a dozen other companies as well. Well talk about some features that make it stand out from other Scala big-data frameworks, including our uses of Algebird, macros, shapeless, magnolia, etc. to make large scale data processing easier, safer, & faster.

========================================

Title: Music Is Just Wiggly Air: Streaming Signal Processing in Python at Scale

Bio:
Lynn Root is an SRE at Spotify with historical issues of using her last name as her username, & the resident FOSS evangelist. She is also a global leader of PyLadies & former Vice Chair of the Python Software Foundation Board of Directors. When her hands are not on a keyboard, they are usually holding a bass guitar or a paint brush.

Abstract:
Digital signal processing (DSP) has been made easy with the help of many Python libraries, allowing engineers & researchers to quickly & effortlessly analyze audio, images, & video. However, scaling these algorithms & models to process millions of files has not been equally as seamless.At Spotify, were trying to address scaling DSP over our catalog of over 50 million songs. This talk will discuss the challenges weve encountered while building the infrastructure needed to support signal processing at scale. Ill discuss the how weve leveraged Apache Beam for streaming data pipelines & the tooling weve built on top of Beam to support our heavy resource requirements.

 
 
 
 
© 2024 GarysGuide      About    Feedback    Press    Terms