The AI Engineer's Guide to Surviving the EU AI Act

Chapter 3. Data and AI Governance and AI Engineering

The first chapter of this book introduced the EU AI Act and laid the foundation for understanding trustworthy AI as its primary motivation. We then explored the engineering aspects of the ML development process and examined the synergy between the CRISP-ML(Q) lifecycle model and the technical components of MLOps, highlighting how this integration supports the development of trustworthy AI through structured processes, transparency, and reproducibility. Here, we will focus on compliance from an organizational perspective, considering the interplay between data and AI governance, risk management, and conformity with the EU AI Act.

The Importance of Data and AI Governance in the EU AI Act Era

Let’s start with three notable real-world examples that illustrate how poor data governance and AI governance practices have led to significant failures of AI products:

IBM Watson for Oncology

The development of Watson for Oncology (launched in 2016) faced significant criticism due to the system recommending unsafe and incorrect cancer treatments. This was attributed to incomplete and biased training data that included only a small number of synthetic cancer cases rather than real clinical data, resulting in flawed recommendations and limited clinical relevance. Internal documents and customer feedback highlighted a lack of transparency, systemic issues, and dissatisfaction with the product’s recommendations. This case emphasizes the critical ...