The Microsoft® Data Warehouse Toolkit: With SQL Server 2008 R2 and the Microsoft® Business Intelligence Toolset, Second Edition

Chapter 9. Design Requirements for Real-Time BI

	"The only reason for time is so that everything doesn't happen at once."
	--— Albert Einstein

What does real time mean in the context of data warehousing and business intelligence? If you ask your business users what they mean when they ask for real-time data, you'll get such a range of answers that you may decide it simply means "faster than they get data today."

Throughout this book we've been assuming the DW/BI system is refreshed periodically, typically daily. All the techniques we've discussed are perfectly appropriate for a daily load cycle. In this chapter, we turn our attention to the problem of delivering data to business users throughout the day. This can be every 12 hours, hourly, or possibly even with a very low latency of seconds.

We'll begin the chapter by confessing that we're not huge fans of integrating real-time data into the data warehouse. This isn't to say we don't think real-time data is interesting — just that putting it in the data warehouse database can be very expensive and may be requested impulsively by end users who haven't made a solid case for real-time data. The best use cases we've heard for very low latency data come from mixed workload operational applications where a customer is on the phone or online.

Putting aside our doubts, and assuming your business users truly require intraday data, we turn our attention to the hard problem: getting low latency data to the business users. Depending on users' requirements, ...