Microsoft PolyBase Hadoop and Blob Storage: Processing Big Data and Cloud Objects

Microsoft PolyBase Hadoop and Blob Storage: Processing Big Data and Cloud Objects

by Kevin Feasel
Released September 2020
Publisher(s): Apress
ISBN: 9781484265840

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Video description

This video dives into Microsoft SQL Server's PolyBase feature for virtualizing data from Apache Hadoop and Azure Blob Storage into SQL Server. From this video you will learn how to configure SQL Server and PolyBase to connect to Azure Blob Storage, including how to make connections to public and secured blob containers. You’ll also learn how to make connections to Hadoop clusters and query data that is stored in Hadoop.

The video begins by showing how to read from and insert data into Azure Blob Storage. From there, you will learn how to connect PolyBase to a Hadoop cluster, whether in Cloudera’s distribution of Hadoop or the Hortonworks Data Platform. Once connected, you will learn how to query data in Hadoop using two methods:  streaming and predicate pushdown. Finally, the video provides key tips on troubleshooting Hadoop integrations, including tips on where to look for log data as well as common configuration and data issues.

What You Will Learn
  • Connect to Azure Blob Storage and Hadoop clusters from SQL Server
  • Uncover configuration options needed to connect to a Hadoop cluster
  • Use predicate pushdown to create MapReduce jobs which offload work to a Hadoop cluster
  • Write data to Hadoop and Azure Blob Storage using T-SQL from SQL Server
  • Troubleshoot common errors when connecting to Hadoop or Azure Blob Storage
Who This Video Is For

Data platform specialists who need to work on multiple data platforms but want to use one unifying language to tie them together. Viewers should be familiar with SQL Server and the T-SQL language, and should be familiar with the content from the prior video in the series, PolyBase Concepts and Configuration.

Publisher resources

Download Example Code

Product information

  • Title: Microsoft PolyBase Hadoop and Blob Storage: Processing Big Data and Cloud Objects
  • Author(s): Kevin Feasel
  • Release date: September 2020
  • Publisher(s): Apress
  • ISBN: 9781484265840