Transformers: The Definitive Guide

Chapter 11. Deploying Transformer Models

Now that you know about different transformer models, it’s time to put them to work. Deploying transformers with tools like vLLM or SGLang has become straightforward on most major cloud platforms, such as GCP or AWS. Therefore, this chapter doesn’t focus on cloud deployment instructions but on the objectives that kept me up at night when I first deployed transformer-based application, such as cost and data control, runtime engineering, cache behavior, architecture-specific constraints, and infrastructure strategy.

With this in mind, one of the first choices that shapes how you set up your application is whether you rely on open or closed models. This choice is not a purely technical question; it determines your organization’s ability to control costs, scale flexibly, audit behavior, customize models, and maintain long-term independence.

It’s also important to understand that deploying a transformer isn’t simply a matter of taking an open source checkpoint and calling .generate. In production, a model becomes just one component in a larger system that includes routing, storage, monitoring, guardrails, and infrastructure. And even if you choose open source models, this does not automatically mean self-hosting. Most open models can also be accessed through managed cloud APIs, allowing fast experimentation before eventually moving to self-hosted deployments for cost efficiency or compliance.

Once you decide how you want to access your model—closed ...