Video description
This video dives into Microsoft SQL Server's PolyBase feature for virtualizing data from Apache Hadoop and Azure Blob Storage into SQL Server. From this video you will learn how to configure SQL Server and PolyBase to connect to Azure Blob Storage, including how to make connections to public and secured blob containers. You’ll also learn how to make connections to Hadoop clusters and query data that is stored in Hadoop.
The video begins by showing how to read from and insert data into Azure Blob Storage. From there, you will learn how to connect PolyBase to a Hadoop cluster, whether in Cloudera’s distribution of Hadoop or the Hortonworks Data Platform. Once connected, you will learn how to query data in Hadoop using two methods: streaming and predicate pushdown. Finally, the video provides key tips on troubleshooting Hadoop integrations, including tips on where to look for log data as well as common configuration and data issues.
What You Will Learn
- Connect to Azure Blob Storage and Hadoop clusters from SQL Server
- Uncover configuration options needed to connect to a Hadoop cluster
- Use predicate pushdown to create MapReduce jobs which offload work to a Hadoop cluster
- Write data to Hadoop and Azure Blob Storage using T-SQL from SQL Server
- Troubleshoot common errors when connecting to Hadoop or Azure Blob Storage
Who This Video Is For
Data platform specialists who need to work on multiple data platforms but want to use one unifying language to tie them together. Viewers should be familiar with SQL Server and the T-SQL language, and should be familiar with the content from the prior video in the series, PolyBase Concepts and Configuration.
Table of contents
- Overview 00:01:02
- What is PolyBase? 00:03:49
- Configuring Azure Blob Storage 00:11:56
- Reading from Azure Blob Storage 00:08:42
- Writing to Azure Blob Storage 00:09:17
- Hadoop Configuration 00:10:11
- Streaming vs Pushdown 00:03:34
- Streaming Data from Hadoop 00:09:11
- Running MapReduce Jobs 00:13:44
- Diagnosing Common Issues 00:16:42
- Conclusion 00:01:24
Product information
- Title: Microsoft PolyBase Hadoop and Blob Storage: Processing Big Data and Cloud Objects
- Author(s):
- Release date: September 2020
- Publisher(s): Apress
- ISBN: 9781484265840
You might also like
book
PolyBase Revealed: Data Virtualization with SQL Server, Hadoop, Apache Spark, and Beyond
Harness the power of PolyBase data virtualization software to make data from a variety of sources …
video
Create an Azure SQL Data Warehouse in Minutes
Microsoft's Azure SQL Data Warehouse is a massively parallel processing (MPP) distributed database system that integrates …
video
SQL Server 2019 Big Data Clusters Crash Course: Installing and Using a Big Data Cluster for Data Analysis
Learn the architecture and implementation of Microsoft’s latest SQL Server capability - Big Data Clusters. Combine …
video
PolyBase in SQL Server 2019: Data Virtualization with SQL Server, Cosmos DB, PostgreSQL, and Other Database Engines
This video dives into PolyBase - Microsoft's technology for data virtualization - and its ability to …
