SNA Projects Blog

S4: Distributed Stream Computing Platform Leo Neumeyer & Anish Nair (Yahoo!) Tuesday, January 25, 2011 ABSTRACT S4 is a general-purpose, distributed, scalable, partially fault-tolerant, pluggable platform that allows programmers to easily develop applications for processing continuous unbounded streams of data. Keyed data events are routed with affinity to Processing Elements (PEs), which consume the events, [...]

Mesos: Efficiently Sharing the Datacenter Benjamin Hindman (Berkeley) Monday, November 8, 2010 ABSTRACT More and more applications are being developed to run primarily in datacenters. Prominent examples include web applications (e.g., using Ruby on Rails) and analytics (e.g., using Hadoop) in addition to the numerous services (e.g., memcached) that support front ends and back ends [...]

Mirage: A New Multi-Scale Operating System for Clouds and Crowds Anil Madhavapeddy (University of Cambridge) Friday, October 22, 2010 ABSTRACT Applications run on all kinds of environments these days: multicore desktops, virtual cloud infrastructures, smart-phones, and web browsers. These diverse environments make it worth rethinking the long-term future of our software stacks; do we really [...]

Akka: Simpler Scalability, Fault-Tolerance, Concurrency & Remoting through Actors Jonas Bonér Thursday, September 23, 2010 ABSTRACT Writing correct concurrent, fault-tolerant and scalable applications is too hard. Most of the time it’s because we are using the wrong tools and the wrong level of abstraction. Akka is here to change that. Akka is using the Actor [...]

Sam Rash from Facebook came by and talked to us about the how they provide near-realtime access to data logged from Scribe into HDFS. Very fascinating. Enjoy! Low Latency Message Bus with Scribe and HDFS Sam Rash (Facebook) Tuesday, August 31, 2010 ABSTRACT This talk covers the Data Freeway project at Facebook, which centers around [...]