In Chapter 8, I discussed a variety of methods for capturing Azure Data Factory pipeline logs and persisting the data either to a SQL table or within Azure Data Lake Storage Gen2. While this process of capturing pipeline log data is valuable when the pipeline activities succeed, this chapter will cover how to capture and persist Azure Data Factory pipeline errors to a SQL table within the ADF_DB that was created in previous chapters. Additionally, we will recap the pipeline ...
© The Author(s), under exclusive license to APress Media, LLC, part of Springer Nature 2021
R. C. L'EsteveThe Definitive Guide to Azure Data Engineeringhttps://doi.org/10.1007/978-1-4842-7182-7_99. Capture Pipeline Error Logs in SQL Database
Ron C. L’Esteve1
(1)
Chicago, IL, USA
Get The Definitive Guide to Azure Data Engineering: Modern ELT, DevOps, and Analytics on the Azure Cloud Platform now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.
