Skip to Main Content
Foundations for Architecting Data Solutions
book

Foundations for Architecting Data Solutions

by Ted Malaska, Jonathan Seidman
September 2018
Beginner to intermediate content levelBeginner to intermediate
187 pages
4h 59m
English
O'Reilly Media, Inc.
Content preview from Foundations for Architecting Data Solutions

Chapter 1. Key Data Project Types and Considerations

The basis for any successful data project is a clear understanding of what you’re tasked to build and then understanding the major items that you need to consider in order to design a solid solution. We categorize data projects into three types that from our experience will typify many data projects. This categorization then allows us to explore the primary items we need to consider before starting on implementation. Not every project will fall neatly into one of these categories, and some projects might straddle these categories, but we feel that these project types will provide a useful framework for understanding your data use cases.

In this chapter, we begin by describing these major project types, followed by a description of the main items to consider, in general, for implementing solutions. We then take a deeper dive into these considerations for each project type.

Major Data Project Types

Let’s begin by describing the three project types that we use to categorize data projects:

Data pipelines and data staging

We can think of these as Extract, Transform, and Load (ETL)–type projects; in other words, these are projects that involve the collection, staging, storage, modeling, and so on of datasets. These are essentially projects that provide the basis for performing subsequent analysis and processing of data.

Data processing and analysis

These are projects that end in providing some kind of actionable value. This might ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Start your free trial

You might also like

Data Engineering with AWS

Data Engineering with AWS

Gareth Eagar

Publisher Resources

ISBN: 9781492038733Errata Page