Essential Cybersecurity Science

Chapter 3. Cybersecurity Experimentation and Test Environments

Scientific inquiry and experimentation require time, space, and materials. Depending on type, scale, cost, and other factors, you may want to run an experiment on your laptop, in a lab, on a cloud, or in the real world. In the checklist for experimentation in Chapter 2, an early step in testing a hypothesis was to “identify the environment or test facility where you will conduct experimentation.” This chapter explores that topic and explains the trade-offs and choices for different types of experimentation.

One way to think about experimentation is in an ecosystem, in other words, the “living” environment and digital organisms. The most obvious ecosystem is the real world. Knowledge about cybersecurity science is certainly gained by observing and interacting with the real world, and some scientists firmly believe that experimentation should start with the real world because it grounds science in reality.

Sometimes the real world is inappropriate or otherwise undesirable for testing and evaluation. It would be unethical, dangerous, and probably illegal to study the effects of malware by releasing it onto the Internet. It is also challenging to observe or measure real-world systems without affecting them. This phenomenon is called the observer effect. Studying the way that users make decisions about cybersecurity choices is valuable, but once subjects know that a researcher is observing them, their behavior changes.

Consider ...