Based on a conversation Oscar Boykin had at NE Scala 2013, he returned to the west coast and started developing Summingbird; a library to represent streaming map/reduce in a type-safe way. The idea came from a challenge to compose a lot map/reduce flows (which is something that comes up often at Twitter), while being compatible on many different platforms, in a more efficient way than allowed by the current tools.
Oscar describes the problems, and eventual solutions, that came up during the development of Summingbird. He also describes how Twitter uses Summingbird to power realtime analytics and consumer facing products. Witness the power of monoids, combinators, and abstract types to build portable streaming aggregations.
There's more from NE Scala 2014!
Follow along as we post each of the talks from this year's Northeast Scala Symposium.