book

ETL with Azure Cookbook

Name: ETL with Azure Cookbook
ISBN: 9781800203310

by Christian Cote, Matija Lah, Madina Saitakhmetova

September 2020

Beginner to intermediate

446 pages

9h 18m

English

Packt Publishing

Read now

Unlock full access

ETL with Azure Cookbook
Why subscribe?ContributorsAbout the authorsAbout the reviewersPackt is searching for authors like you
Preface
Who this book is forWhat this book coversTo get the most out of this bookDownload the example code filesDownload the color imagesConventions usedSectionsGetting readyHow to do it…How it works…There's more…See alsoGet in touchReviews
Chapter 1: Getting Started with Azure and SSIS 2019
Technical requirementsInstalling Microsoft SQL Server 2019 Integration ServicesGetting readyHow to do it…How it works…Installing SQL Server Management StudioGetting readyHow to do it…How it works…Installing SQL Server Data ToolsGetting readyHow to do it…How it works…Installing Azure Data StudioGetting readyHow to do it…How it works…Creating an Azure subscriptionGetting readyHow to do it…How it works…There's more…
Chapter 2: Introducing ETL
Creating a SQL Azure databaseGetting readyHow to do it…How it works…Connecting SQL Management Studio to Azure SQLGetting readyHow to do it…How it works…Creating a simple ETL packageGetting readyHow to do it…How it works…Loading data before its transformationGetting readyHow to do it…How it works…
Chapter 3: Creating and Using SQL Server 2019 Big Data Clusters
Technical requirementsCreating a SQL Server 2019 Big Data ClusterGetting readyHow to do it…How it works…Stopping and starting Azure resourcesGetting readyHow to do it…How it works…Setting up a Big Data ClusterGetting readyHow to do it…How it works…Loading data into a Big Data ClusterGetting readyHow to do it…How it works…Extracting data from a Big Data ClusterGetting readyHow to do it…How it works…
Chapter 4: Azure Data Integration
Setting up the Azure Feature Pack in SSIS Getting readyHow to do it…How it works…Setting up a Java Runtime EnvironmentGetting readyHow to do it…How it works…Setting up an Azure storage accountGetting readyHow to do it…How it works…Creating an on-demand Azure HDInsight clusterGetting readyHow to do it…How it works…Transforming data with HiveGetting readyHow to do it…How it works…Transforming data with PigGetting readyHow to do it…How it works…
Chapter 5: Extending SSIS with Custom Tasks and Transformations
Technical requirementsDesigning a Custom Control Flow TaskGetting readyHow to do it…How it works…Designing a Custom Data Flow ComponentGetting readyHow to do it…How it works…There's more…Deploying SSIS customizationsGetting readyHow to do it…How it works…Upgrading a Custom Data Flow ComponentGetting readyHow to do it…How it works…
Chapter 6: Azure Data Factory
Data factory creationGetting readyHow to do it…How it works…Copying data from the internetGetting readyHow to do it…How it works…Moving and transforming dataGetting readyHow to do it…How it works…Triggering and monitoring our pipelineGetting readyHow to do it…How it works…
Chapter 7: Azure Databricks
Creating a Databricks workspaceGetting readyHow to do it…How it works…Creating a cluster in our workspaceGetting readyHow to do it…How it works…Connecting to an internet data source Getting readyHow to do it…How it works…Using Delta LakeGetting readyHow to do it…How it works…There's more…Writing in Azure SQL ServerGetting ready How to do it…How it works…
Chapter 8: SSIS Migration Strategies
Running SSIS packages in ADFGetting readyHow to do it…How it works…Rewriting an SSIS package using ADFGetting readyHow to do it…How it works…

Chapter 9: Profiling data in Azure
Technical requirementsUsing SQL in SparkGetting readyHow to do it…How it works…Using Spark built-in functionsGetting readyHow to do it…How it works…Using pandas profilingGetting readyHow to do it…How it works…
Chapter 10: Manage SSIS and Azure Data Factory with Biml
Technical requirementsInstalling prerequisitesHow to do it…How it works…Generating T-SQL select statements for all tablesGetting readyHow to do it…How it works…There's more…Generating T-SQL to drop and create all indexesGetting readyHow to do it…How it works…Generating a basic SSIS packageGetting readyHow to do it…How it works…There's more…Generating an SSIS package with Execute SQL TasksGetting readyHow to do it…How it works…There's more…Using Convert SSIS package to BimlGetting readyHow to do it…How it works…There's more…Generating a mass change to stored proceduresGetting readyHow to do it…How it works…There's more…Generating an SSIS package containing a Data Flow TaskGetting readyHow to do it…How it works…Generating your first Azure Data FactoryGetting readyHow to do it…How it works…There's more…Loading multiple files using Data FactoryGetting readyHow to do it…How it works…See also
Other Books You May Enjoy
Leave a review - let other readers know what you think

Overview

ETL with Azure Cookbook is a comprehensive guide to building effective and scalable ETL solutions using the Azure cloud platform. Through hands-on recipes, this book explores the features and capabilities of Azure services for data integration and transformation, guiding you in creating efficient processes for moving and handling data.

What this Book will help me do

Master the basics and advanced techniques for building ETL processes on Azure.
Learn practical skills in designing solutions that integrate multiple Azure services.
Understand how to migrate existing on-premises ETL solutions to Azure successfully.
Acquire knowledge of SQL Server and Azure Big Data Clusters for data integration.
Gain experience in automating and optimizing data processes with BIML and Azure Databricks.

Author(s)

The authors of ETL with Azure Cookbook are experienced data engineers and Azure specialists with years of expertise in designing and implementing robust data solutions. Their professional journey includes hands-on work with SQL Server, Azure services, and scalable ETL frameworks. They aim to provide practical insights and actionable guidance to help readers achieve success in data engineering projects.

Who is it for?

This book is ideal for data architects, ETL developers, and IT professionals seeking to enhance their skills in data integration and transformation, particularly within the Azure ecosystem. It's suitable for individuals with some knowledge of data engineering principles, SQL, and familiarity with ETL processes who aim to adopt modern cloud-based approaches.

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Publisher Resources

ISBN: 9781800203310

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills