Table of Contents
Preface
Section 1: Building Data Pipelines – Extract Transform, and Load
Chapter 1: What is Data Engineering?
What data engineers do 4
Required skills and knowledge to be a data engineer 6
Data engineering versus data science 7
Data engineering tools 7
Programming languages 8
Databases 8
Data processing engines 10
Data pipelines 11
Summary 15
Chapter 2: Building Our Data Engineering Infrastructure
Installing and configuring Apache NiFi 18
A quick tour of NiFi 20
PostgreSQL driver 27
Installing and configuring Apache Airflow 27
Installing and configuring Elasticsearch 34
Installing and configuring Kibana 36
Installing and configuring PostgreSQL 41
Installing pgAdmin 4 41
A tour of pgAdmin 4 42
Summary 44 ...
Get Data Engineering with Python now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.
