LLMOps

Chapter 6. API-First LLM Deployment

Choosing the right tools for deploying LLMs can make or break your project.

Open source tools give you more control but require you to do more work, while managed services are easier to set up and scale but often come at a higher cost. A popular repository of open source tools and data is HuggingFace, which contains a lot of pretrained models and tools to help with tasks like tokenization, fine-tuning, and data processing.

The business model you choose will impact your revenue, costs, and user experience and, thus, also your deployment decision. By understanding your users’ needs, evaluating your costs, and considering your competition, you can choose a business model that meets your needs and provides value to your users. Options include:

Infrastructure as a service (IaaS)

This model is suitable for organizations that want to build and deploy their own LLM applications but don’t want to manage the underlying infrastructure.

With IaaS, organizations can provision and configure computing resources quickly and easily, without the need for significant up-front investment. It provides flexibility and control over the infrastructure, allowing organizations to customize and optimize the environment for their specific needs.

IaaS is a good fit for organizations that have the expertise and resources to manage their own applications and infrastructure. However, it requires a higher level of technical expertise and management than do other business ...