Dependable Computing

14The Future: Dependable and Trustworthy AI Systems

14.1 Introduction

The emergence of artificial intelligence (AI) systems and their ubiquitous adoption in automating tasks that involve humans in critical application domains (e.g. autonomous vehicles (AVs), medical assistants/devices, manufacturing, agriculture, and smart buildings) means that it is of paramount importance that we are able to place trust in these technologies.

In a broad sense, a trustworthy AI system must be both dependable (i.e. ensure the safety, resilience, robustness, and security of both itself and its operational environment) and explainable (i.e. provide the reasoning behind the decisions/actions produced). The absence of such features remains a major impediment to the public acceptance of these technologies and their deployment in human‐ and mission‐critical applications, despite successful demonstrations. Also, there are ongoing concerns, from both algorithmic and experimental perspectives, regarding the reliability, safety, and security of machine learning (ML) technologies and systems that incorporate ML technologies, and whether such systems may be vulnerable to security hacks and crashes that can ultimately impact human safety. As Schneier states in [57], traditionally, computers have outperformed humans only with respect to speed, scale, and scope, whereas humans have excelled at thinking, reasoning, adapting, and understanding. However, AI is changing the landscape, such that computers can ...