Introducing DataFrames in Spark for Large Scale Data Science

I'm the author of this blog post. We are very excited about this API and think it will be the common interchange format for data in Spark. It has also some neat features (such as code generation, predicate push down, etc) that make it very useful for Big Data. Feel free to ask me anything.