Designing Large Language Model Applications

Chapter 7. Interfacing LLMs with External Tools

A Note for Early Release Readers

With Early Release ebooks, you get books in their earliest form—the author’s raw and unedited content as they write—so you can take advantage of these technologies long before the official release of these titles.

This will be the 8th chapter of the final book. Please note that the GitHub repo will be made active later on.

If you have comments about how we might improve the content and/or examples in this book, or if you notice missing material within this chapter, please reach out to the author at [e-mail].

In order to effectively harness the power of LLMs in your organization, they have to be integrated into the existing data and software ecosystem. Unlike traditional software components and data stores, LLMs can generate autonomous actions to interact with other components of the ecosystem, thus bringing a degree of flexibility never seen before in the world of software. This flexibility unlocks a whole host of use cases that were previously considered impossible.

There is another reason why we need LLMs to interact with software and external data. As we know too well, current LLMs have significant limitations, some of which we discussed in Chapter 1. To recap some key points:

• Since it is expensive to retrain LLMs or keep them continuously updated, they have a knowledge cutoff date and thus possess no knowledge of more recent events.

• Most LLMs perform poorly at mathematical operations ...