Chapter 1

Understand Azure data solutions

Data engineers are responsible for data-related implementation tasks. These include the following:

  • Provisioning data-storage services

  • Ingesting streaming and batch data

  • Transforming data

  • Implementing security requirements

  • Implementing data-retention policies

  • Identifying performance bottlenecks

  • Accessing external data sources

Mastering data engineering requires learning a specific set of tools, architectures, and platforms. Where the SQL professional, for example, might master and use T-SQL as the primary data-manipulation tool, the data engineer may use additional technologies (like HDInsight and Cosmos DB) and languages (such as Python) to manipulate data in big data systems.

Additionally, moving ...

Get Microsoft Azure Data Solutions - An Introduction now with O’Reilly online learning.

O’Reilly members experience live online training, plus books, videos, and digital content from 200+ publishers.