9
The Photon Engine
In this chapter, we will turn our attention back to SQL Warehouses. This time, however, we will focus on the query engine running on SQL Warehouses. The query engine is known as Photon Engine. We will begin by learning about Photon Engine and its place in the Apache Spark framework. Going ahead, we will understand the core engineering philosophy of Photon Engine. Finally, we will go through its limitations and the roadmap to overcome them.
I do want to highlight that you don’t need to learn the details of Photon Engine to work with Databricks SQL. This chapter is intended for those who are interested in how Databricks SQL achieves record-beating query performances on the Lakehouse setup with open source storage formats.
Get Business Intelligence with Databricks SQL now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.
