In this chapter, you will modify your “Hello, CDK” application to form the foundation of the Home Energy Coach application. You will create CDK stacks for shared resources and the data pipeline. You will build upon your existing application by repurposing your S3 bucket and Lambda function as a data pipeline. This pipeline will allow you to upload home energy usage data and receive a summary of total energy consumed. You can either use a provided sample dataset or fill in the data template with information from your own utility bill. This chapter will prepare you for the next one, where you will learn multiple ways to package Lambda functions.
The projects are going to get progressively more involved, starting here. There is a jump from the earlier intro chapters to what you will make in this chapter. We know it is tough, but take your time, and we are confident you will be able to continue building cool stuff.
Project Architecture: Data Pipeline Microservice
In this project segment, you will build a data pipeline microservice that allows a user to upload a .csv file containing home energy usage data to an S3 bucket. The system uses Amazon EventBridge to trigger an AWS Lambda function, which then summarizes the total energy used in kilowatt hours (kWh). Figure 5-1 illustrates the application flow up to this point. We use EventBridge to take specific actions based on the incoming data.
Figure 5-1. Serverless data pipeline architecture ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month, and much more.
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.
Julian F.
Head of Cybersecurity
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.
Addison B.
Field Engineer
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.
Amir M.
Data Platform Tech Lead
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.
