Real-Time Intelligence with Microsoft Fabric

Chapter 5. Fabric’s Eventstream Service

When you work with data for analytics, you typically follow distinct phases: ingest, store, process, and serve. These are also the capabilities we consider when designing data platforms. If you come from a data warehousing background, this typically translates to an extract, load, and transform (ELT) pattern. This works well with batch processing of bounded datasets—data with a clear start and endpoint—using tools like Data Factory in Microsoft Fabric or SQL Server Integration Services (SSIS).

However, working with data streams presents fundamentally different challenges that demand specialized approaches and tools. Unlike batch data, streams—or unbounded data—are continuous flows of information with no predetermined start or endpoint. While events typically arrive in chronological order, network latency, system failures, and distributed sources can cause out-of-order delivery, late-arriving data, and duplicates.

Eventstream is the real-time data ingestion, processing, and routing service on the Microsoft Fabric data platform. It is built on mature Azure streaming technologies, and it provides a unified experience for handling continuous dataflows end-to-end, while removing some of the underlying complexity.

In this chapter, we’ll start by discussing the foundational concepts and challenges of stream processing. With that context in mind, we’ll then discuss how Eventstream solves those challenges. We’ll give you an overview of Eventstream ...