March 2025
Intermediate to advanced
48 pages
1h 5m
English
Content preview from AI Gateways in the Enterprise
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
Chapter 3. Common AI Gateway Use Cases
In the previous chapter, we introduced the concept of an AI gateway and explained why traditional networking falls short. This chapter will explore how enterprises can use an AI gateway to solve real-world challenges.
As we explore these use cases, we’ll reference an AI gateway implementation as an example. Users should choose an AI gateway built on modern architectures including Envoy Proxy and Kubernetes Gateway API. Gateways like Gloo AI Gateway (being donated to Cloud Native Computing Foundation—CNCF—as KGateway) enable AI features that accelerate AI application development while addressing critical security, observability, control, and governance needs.
Choosing a reliable AI gateway helps address the top concerns that enterprises have when adopting AI, including:
-
Security threat mitigation
-
The bridging of the technical skills gap
-
Seamless integration with existing infrastructure
With this context in mind, let’s explore the key use cases that make AI gateways essential for enterprise AI adoption.
Security and Access Control
Security and access control are paramount for every technology, and AI use cases aren’t any different. An AI gateway can encompass everything from managing API credentials to multi-tenant isolation.
In this section, we’ll explore how an AI gateway provides a security layer that handles authentication, authorization, and access policies to ensure that AI resources and LLM providers are accessed only by authorized ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.