Bloomberg is a technology company specializing and leading in providing data solutions to people in the financial industry. Of particular significance amongst functionalities Bloomberg offers is a real-time news search and alerting system which offers it's customers information in a timely manner. While providing a real-time search engine with low latency and even lower index to search latency is a challenge in itself, what makes Bloomberg's offering unique is real-time alerting on any arbitrarily complex boolean query supported by the search engine. So what does it take to build an alerting engine which stores millions of query subscriptions changing every second, and matches them against hundreds of stories coming in every second at peak, to dynamically update user's screens, e-mail them, show popups or otherwise notify them when a news story matches their interest? And what does it take to do this all this in less than 100ms? In this talk, Daniel Collins of the News Search backend team dives into the alerting backend his team maintains, and the story of how they rewrote this entire alerting infrastructure using and enhancing Lucene, and Luwak, a library for managing stored queries.

Bio:

Daniel is a member of the News Search backend team at the Bloomberg R&D office in London. He joined Bloomberg in 2011,after 12 years working in R&D in the telecoms space working on both fixed and mobile telecoms solutions. He started working with Apache Solr/Lucene in 2012, and has led the effort on migrating news' indexing and alerting pipeline from the existing infrastructure to using Lucene, Luwak and Solr.