System Design on AWS

Chapter 20. Designing a Video-Processing Pipeline for a Streaming Service

We talked about video uploads and downloads between users on a messaging application in Chapter 19. To ensure optimal customer experience, there are certain steps we need to take in this video-uploading process, such as compression to reduce the size and transcoding to support different media formats and a wide variety of devices. In this chapter, we’ll focus on building a video-processing pipeline for streaming services like Netflix, Amazon Prime Video, and YouTube. As always, we’ll start with the system requirements and then dive into the details of the different steps involved. The main goal of our design decisions throughout the chapter will be for the video content to be available to viewers in the optimal quality and the minimum time possible. We’ll conclude the chapter by deploying the system on the AWS cloud using AWS media services, and we’ll also touch on some considerations for live streaming large-scale events, such as Thursday Night Football in the US and cricket matches in India.

System Requirements

The most important requirement of the system is that the video uploaded from production houses (in the case of Netflix or Amazon Prime Video) or users (in the case of YouTube) should be processed in such a way that it is easily accessible to viewers later. The system design will be applicable to any system involving video processing, but we’ll use examples from Netflix’s system.

There are a few ...