Building Agentic AI: Workflows, Fine-Tuning, Optimization, and Deployment

6

Moving Beyond Natural Language: Multimodal and Coding AI

Introduction

It’s no secret that chatbots can go quite far on text alone, especially with tools that give them the ability to look things up on the internet for us (just look at the early success of ChatGPT and Claude). But when it comes to truly being transformative, natural language, text-only AI has clear limits. Most of what makes up the internet—and honestly, the real world—isn’t text. It’s images, videos, code (which is text but not really “natural language”), audio, and, increasingly, a mix of all of the above. So, if we want our AI applications to interact with the world or even just a photo roll, we need to move beyond natural language and build systems that can see, hear, ...