Generative AI Application Integration Patterns

7

Integration Pattern: Real-Time Intent Classification

In previous chapters, we discussed the batch-processing integration pattern, where we focused on efficiently processing large volumes of data and generating data to be used by downstream systems. In this chapter, we will shift our focus to real-time integration patterns.

Real-time interactions require applications to be optimized for latency, rather than processing large batch requests efficiently. In other words, we need to ensure that the output is generated as quickly as possible to provide an optimized user experience. The most common use case for this pattern is real-time agents exposed through chat or voice interfaces.

Let’s consider an intent classification use case, which is a common ...