book

Azure OpenAI Service for Cloud Native Applications

by Adrián González Sánchez

July 2024

Beginner to intermediate

248 pages

8h 9m

English

O'Reilly Media, Inc.

Book available

Read now

Unlock full access

Includes

Includes Quizzes

How This Book Is OrganizedConventions Used in This BookUsing Code ExamplesO’Reilly Online LearningHow to Contact UsAcknowledgments
What Is Artificial Intelligence?Current Level of AI AdoptionThe Many Technologies of AITypical AI Use CasesTypes of AI Learning ApproachesAbout Generative AIPrimary Capabilities of Generative AIRelevant Industry ActorsThe Key Role of Foundation ModelsRoad to Artificial General Intelligence Microsoft, OpenAI, and Azure OpenAI ServiceThe Rise of AI CopilotsAzure OpenAI Service Capabilities and Use CasesLLM Tokens as the New Unit of MeasureConclusion
Modernizing Applications for Generative AICloud Native Development with Azure OpenAI ServiceMicroservice-Based Apps and ContainersServerless WorkflowsAzure-Based Web Development and CI/CDUnderstanding the Azure PortalGeneral Azure OpenAI Service ConsiderationsAvailable Azure OpenAI Service ModelsArchitectural Elements of Generative AI SystemsConclusion
Defining the Knowledge Scope of Azure OpenAI Service–Enabled AppsGenerative AI Modeling with Azure OpenAI ServiceAzure OpenAI Service Building BlocksPotential Implementation ApproachesApproach Comparison and Final RecommendationAI Performance Evaluation MethodsConclusion
Plug-insLLM Development, Orchestration, and IntegrationLangChainSemantic KernelLlamaIndexBot FrameworkPower Platform, Microsoft Copilot, and AI BuilderDatabases and Vector StoresVector Search from Azure AI SearchVector Search from Cosmos DBAzure Databricks Vector SearchRedis Databases on AzureOther Relevant Databases (Including Open Source)Additional Microsoft Building Blocks for Generative AIAzure AI Document Intelligence (formerly Azure Form Recognizer) for OCRMicrosoft Fabric’s LakehouseMicrosoft Azure AI SpeechMicrosoft Azure API ManagementOngoing Microsoft Open Source and Research ProjectsConclusion
The Art of Prompt EngineeringGenerative AI and LLMOpsPrompt Flow and Azure MLSecuring LLMsManaging Privacy and ComplianceResponsible AI and New RegulationsRelevant Regulatory Context for Generative AI SystemsCompany-Level AI Governance ResourcesTechnical-Level Responsible AI ToolsConclusion
Premortem, or What to Consider Before Implementing a Generative AI ProjectDefining Implementation Approach, Resources, and Project RoadmapDefining Project WorkstreamsIdentifying Required ResourcesEstimating Duration and EffortCreating a “Living” RoadmapCreating Usage ScenariosCalculating Cost and Potential ROIConclusion
What’s Next? The Evolution Toward Microsoft CopilotExpert Insights for the Generative AI EraDavid Carmona: AI Adoption and the Future of Generative AIBrendan Burns: The Role of Cloud Native for Generative AI DevelopmentsJohn Maeda: About AI Design and OrchestrationSarah Bird: Responsible AI for LLMs and Generative AITim Ward: The Impact of Data Quality on LLM ImplementationsSeth Juarez: From Generative AI Models to a Full LLM PlatformSaurabh Tiwary: The New Microsoft Copilot EraConclusion
Relevant O’Reilly Books for Your Upskilling JourneyOther Resources and Repositories

Content preview from Azure OpenAI Service for Cloud Native Applications

Chapter 4. Additional Cloud and AI Capabilities

Generative AI applications are way more than “just a big model.” As you have already seen, LLMs play a central role, but there are other relevant pieces that complement the capabilities of Azure OpenAI Service: fine-tuning via Azure OpenAI APIs or playgrounds, grounding with Azure Cognitive Search and/or Azure OpenAI embeddings, live search capabilities with Bing Search API, etc.

Additionally, we have new kinds of tools that allow us to expand the capacities of LLMs even more. A curated selection for any generative AI and Azure OpenAI adopter could include plug-ins, LMM integration, databases, and more. Let’s dig into these in more detail.

Plug-ins

One of the most important new terms in AI applications is the notion of “plug-ins.” We can define them as direct interfaces to advanced functionalities, interconnecting Microsoft’s Azure OpenAI (or OpenAI’s ChatGPT) with other systems. For example, there are plug-ins from companies such as Expedia, FiscalNote, Instacart, KAYAK, Klarna, Milo, OpenTable, Shopify, Slack, Speak, Wolfram, and Zapier. They are external to Azure OpenAI, and their nature and business models depend on the developer companies.

Additionally, Microsoft announced in May 2023 its own collection of plug-ins, defining them as “standardized interfaces that allow developers to build and consume APIs to extend the capabilities of large language models (LLMs) and enable a deep integration of GPT across Azure and the Microsoft ...