Chapter 1

Understand Azure data solutions

Data engineers are responsible for data-related implementation tasks. These include the following:

  • Provisioning data-storage services

  • Ingesting streaming and batch data

  • Transforming data

  • Implementing security requirements

  • Implementing data-retention policies

  • Identifying performance bottlenecks

  • Accessing external data sources

Mastering data engineering requires learning a specific set of tools, architectures, and platforms. Where the SQL professional, for example, might master and use T-SQL as the primary data-manipulation tool, the data engineer may use additional technologies (like HDInsight and Cosmos DB) and languages (such as Python) to manipulate data in big data systems.

Additionally, moving ...

Get Microsoft Azure Data Solutions - An Introduction now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.