INTERACTION DESIGN: beyond human-computer interaction, 3rd Edition

Chapter 14

EVALUATION STUDIES: FROM CONTROLLED TO NATURAL SETTINGS

• 14.1 Introduction
• 14.2 Usability Testing
• 14.3 Conducting Experiments
• 14.4 Field Studies

Objectives

The main aims of this chapter are to:

• Explain how to do usability testing.
• Outline the basics of experimental design.
• Describe how to do field studies.

14.1 Introduction

Imagine you have designed a new shared web space intended for advertising second-hand goods. How would you find out whether householders would be able to use it to find what they wanted and whether it was a reliable and effective service? What evaluation methods would you employ?

In this chapter we describe evaluation studies that take place in a spectrum of settings, from controlled laboratories to natural settings. Within this spectrum we focus on usability testing which takes place in usability labs; experiments which take place in research labs; and field studies which take place in natural settings such as people's homes, work, and leisure environments.

14.2 Usability Testing

The usability of products has traditionally been tested in controlled laboratory settings. This approach emphasizes how usable a product is. It has been most commonly used to evaluate desktop applications, such as websites, word processors, and search tools. Doing usability testing in a laboratory, or a temporarily assigned controlled environment, enables evaluators to control what users do and to control environmental and social influences that might impact the users, ...