Deep dive into Project Tungsten: Bring Spark closer to bare metal
Date: This event took place live on September 03 2015
Presented by: Josh Rosen
Duration: Approximately 60 minutes.
Questions? Please send email to
Hosted By: Ben Lorica
Project Tungsten focuses on substantially improving the efficiency of memory and CPU for Spark applications, to push performance closer to the limits of modern hardware. This effort includes three initiatives:
Project Tungsten will be the largest change to Spark’s execution engine since the project's inception. In this talk, we will give an update on its progress and dive into some of the technical challenges we are solving.
About Josh Rosen
Josh Rosen is a Spark Committer working at Databricks. Previously, he was a grad student at UC Berkeley, studying databases and distributed systems. He has been instrumental in the design and implementation of many Spark components, including PySpark and Tungsten.
About Ben Lorica
Ben Lorica is the Chief Data Scientist and Director of Content Strategy for Data at O'Reilly Media, Inc.. He has applied Business Intelligence, Data Mining, Machine Learning and Statistical Analysis in a variety of settings including Direct Marketing, Consumer and Market Research, Targeted Advertising, Text Mining, and Financial Engineering. His background includes stints with an investment management company, internet startups, and financial services. He is an advisor to Databricks.