Model Driven Engineering for Distributed Real-Time Embedded Systems 2009: Advances, Standards, Applications and Perspectives

Chapter 4

Symbolic Execution-Based Techniques for Conformance Testing ¹

4.1. Context

4.1.1. Conformance testing: an introduction

Testing is a widely used validation technique to increase software quality. It consists of executing the System Under Test (SUT) for some particular use cases and in evaluating whether or not corresponding SUT’s executions conform to some requirements. For that, one has to define input test data to be submitted to the SUT and a decision procedure (called oracle) allowing to assign verdicts depending on SUT’s computations. Selecting a set of test data which is both qualitatively and quantitatively sufficient to get confidence in the testing process, is difficult and time consuming. Moreover manually computing test data and assigning verdicts are both long and error prone. Therefore automation is highly desirable. In the framework of model-based development, models can help one to automate both test generation and verdict production. We focus on a particular way to do so, which is called model-based testing or black box testing. Test data are extracted from models, while SUT is seen as a black box with which a tester can interact through an interface. In the case of reactive systems, interactions consist in sending inputs and observing SUT’s reactions (i.e. outputs). Oracles are defined from sets of behaviors specified in models, and so-called conformance relations which define the conformity of interactions observed during the testing process with regard ...