Video description
Project Tungsten focuses on substantially improving the efficiency of memory and CPU for Spark applications, to push performance closer to the limits of modern hardware. This effort includes three initiatives:1. Memory Management and Binary Processing: leveraging application semantics to manage memory explicitly and eliminate the overhead of JVM object model and garbage collection2. Cache-aware computation: algorithms and data structures to exploit memory hierarchy3. Code generation: using code generation to exploit modern compilers and CPUsProject Tungsten will be the largest change to Spark’s execution engine since the project's inception. In this talk, we will give an update on its progress and dive into some of the technical challenges we are solving.
Publisher resources
Table of contents
Product information
- Title: Deep Dive into Project Tungsten: Bring Spark Closer to Bare Metal
- Author(s):
- Release date: September 2015
- Publisher(s): O'Reilly Media, Inc.
- ISBN: 9781491954843
You might also like
book
Apache Spark 2.x Cookbook
Over 70 recipes to help you use Apache Spark as your single big data computing platform …
book
SQL Server 2005 Practical Troubleshooting: The Database Engine
Never-Before-Published Insiders’ Information for Troubleshooting SQL Server 2005. This is the definitive guide to troubleshooting the …
video
Apache Spark 2.0
In March 2016 at Strata in San Jose, CA, a standing room only audience of excited …
book
Scala and Spark for Big Data Analytics
Harness the power of Scala to program Spark and analyze tonnes of data in the blink …