Vision Language Models

Chapter 1. Introduction to Vision and Language

A Note for Early Release Readers

With Early Release ebooks, you get books in their earliest form—the authors’ raw and unedited content as they write—so you can take advantage of these technologies long before the official release of these titles.

This will be the 1st chapter of the final book. Please note that the GitHub repo will be made active later on.

If you’d like to be actively involved in reviewing and commenting on this draft, please reach out to the editor at [e-mail].

We are social and visual animals. We evolved in groups, depending on one another for survival. Our brains are wired for communication, empathy, and connection. Even our emotional well-being is deeply tied to our relationships.

To strengthen our bonds, we developed language, a tool to optimize communication, and, soon after, writing systems that allowed us to reach beyond the limits of face-to-face interaction.

Yet, among all our senses, vision dominates our perception of the world. A huge part of our brain is dedicated to processing visual information. We rely on sight to understand our environment, navigate spaces, recognize others, and pick up subtle social cues like facial expressions and body language.

It’s no surprise, then, that when we set out to build intelligent systems, we were captivated by two grand challenges: teaching them language to communicate with us, and vision to perceive the world as we do.

At first, our systems were limited ...