© Robert Ilijason  2020
R. IlijasonBeginning Apache Spark Using Azure Databrickshttps://doi.org/10.1007/978-1-4842-5781-4_10

10. Running in Production

Robert Ilijason1 
(1)
Viken, Sweden
 

If what you developed is good and not a one-off, you’ll want to put it into production and have it run on a schedule. Valuable results should be delivered on a regular basis, possibly to another piece of software that’ll execute on the findings. This topic is huge, and we’ll only scratch the surface here by focusing on technical parts in Databricks.

You will want to run your code autonomously when it’s time for production, so we’ll spend a bit of time looking at jobs. We’ll then dive into the built-in solution for running notebooks on a defined schedule.

While most ...

Get Beginning Apache Spark Using Azure Databricks: Unleashing Large Cluster Analytics in the Cloud now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.