Designing with Data

Chapter 3. A Framework for Experimentation

IN THIS CHAPTER, WE move from the theoretical to the practical by introducing the framework that we’ll be using over the next three chapters to dive into the practical integration of data and design. This framework is our own particular take on how to think about A/B testing. You’ll find that many people use similar ways of thinking about A/B testing design and implementation, and we hope that you will find your own ways to adapt our framework to suit your needs. There are three phases that we’ll go into more detail on: definition, execution, and analysis. We’ll dedicate a chapter to each of these and go into much more detail about applying the concepts in Chapter 4–Chapter 6.

What is the relationship between data, design, and user experience? We say that there should be no divide between data insight and design insight. We believe designers have a significant role to play in the design of data-gathering activities (what should be captured and why), in the design of data itself (what format should data take, what data types need to be triangulated), and in the presentation and communication of insights and results that derive from the data capture and analysis. We believe that data capture, data validation, data curation and management, and data analysis for insights are central to design practice. We believe that design practice is central to effective data design.

Furthermore, a user experience that is useful, usable, and elegant—and that ...