LLMs and Generative AI for Healthcare

Chapter 2. Peeking Inside the AI Black Box

As large language models (LLMs) and generative AI tools filter into healthcare applications, they bring along their complex, opaque elements—the inherent “black box” quality that makes the internal workings of these systems seem obscure. This raises natural questions for clinicians and healthcare leaders who are contemplating using LLMs. How exactly do these AI systems develop their clinical competencies? What transpires behind the scenes during their training? Does the black box nature of these emerging tools render LLMs’ recommendations and thought processes too inscrutable for serious medical or healthcare use?

Chapter 2 lifts the hood to examine what resides within the AI systems making today’s headlines. We discuss how transformers, self-attention mechanisms, neural networks, and other technical elements ingest healthcare knowledge and develop reasoning abilities. While a full accounting of the mathematics behind each component is beyond a typical reader’s needs, we provide an accessible explanation of how LLMs work. Peeking inside the black box dispels notions of “magic” while bringing responsible AI adoption within reach, even amid lingering opacity.

LLMs and Generative AI

LLMs use deep learning, a subset of machine learning, and uses layers of algorithms to process data and imitate the human thinking process. The term deep learning and neural network are used interchangeably because all deep learning systems comprise neural networks. ...