Prompt Engineering for LLMs

Chapter 11. Looking Ahead

Human history only makes sense on a logarithmic scale. It took humans countless eons to figure out farming, millennia beyond that to invent writing, centuries more to invent the steam engine, and decades more to invent the automobile, computer, and smartphone. Just a few years after that, around 2012, deep learning appeared on the scene.

OpenAI’s GPT-2 was announced in 2019, and then ChatGPT was announced in 2022. This ignited an explosion of development around LLMs. Many companies have jumped into the fray—Anthropic, Google, Microsoft, Meta, xAI, NVIDIA, Mistral, and more—all building new LLMs that have leap-frogged the previous ones in capability, capacity, and speed. In mere months, LLMs have morphed from document completion engines, to chat engines, to agents that can interact with the outside world.

Buckle up, readers. If you think the pace of change is fast now, then just wait, it’s only going to get faster. (Maybe that Ray Kurzweil guy was on to something!) In this final chapter, let’s look ahead to some of the developments on our horizon and how they will change your work as a prompt engineer.

Multimodality

There is a huge push toward the use of multimodal models. OpenAI kicked off this trend with GPT-4, which was able to process images as part of the prompt. Although OpenAI has not disclosed details about how exactly the model works, most likely, it closely follows the methods published in academic literature.

In one such method, a convolutional ...