book

Adversarial AI Attacks, Mitigations, and Defense Strategies

Name: Adversarial AI Attacks, Mitigations, and Defense Strategies
Author: John Sotiropoulos
ISBN: 9781835087985

by John Sotiropoulos

July 2024

Intermediate to advanced

602 pages

16h 31m

English

Packt Publishing

Read now

Unlock full access

Adversarial AI Attacks, Mitigations, and Defense Strategies
ContributorsAbout the authorAbout the reviewersDisclaimer
Preface
Who this book is forWhat this book coversTo get the most out of this bookDownload the example code filesConventions usedGet in touchShare your thoughts
Part 1: Introduction to Adversarial AI
Chapter 1: Getting Started with AI
Getting the most out of this book – get to know your free benefits Interactive AI assistant (beta)DRM-free PDF or ePub version Understanding AI and MLTypes of ML and the ML life cycleKey algorithms in MLNeural networks and deep learningML development toolsSummaryFurther reading
Chapter 2: Building Our Adversarial Playground
Technical requirementsSetting up your development environmentPython installationCreating your virtual environmentInstalling packagesRegistering your virtual environment with Jupyter notebooksVerifying your installationHands-on basic baseline MLSimple NNsDeveloping our target AI service with CNNsSetup and data collectionData explorationData preprocessingAlgorithm selection and building the modelModel trainingModel evaluationModel deploymentInference serviceML development at scaleGoogle ColabAWS SageMakerAzure Machine Learning servicesLambda Labs CloudSummary
Chapter 3: Security and Adversarial AI
Technical requirementsSecurity fundamentalsThreat modelingRisks and mitigationsDevSecOpsSecuring our adversarial playgroundHost securityNetwork protectionAuthenticationData protectionAccess controlSecuring code and artifactsSecure codeSecuring dependencies with vulnerability scanningSecret scanningSecuring Jupyter NotebooksSecuring models from malicious codeIntegrating with DevSecOps and MLOps pipelinesBypassing security with adversarial AIOur first adversarial AI attackTraditional cybersecurity and adversarial AIAdversarial AI landscapeSummary
Part 2: Model Development Attacks
Chapter 4: Poisoning Attacks
Basics of poisoning attacksDefinition and examplesTypes of poisoning attacksPoisoning attack examplesWhy it mattersStaging a simple poisoning attackCreating poisoned samplesBackdoor poisoning attacksCreating backdoor triggers with ARTPoisoning data with ARTHidden-trigger backdoor attacksClean-label attacksAdvanced poisoning attacksMitigations and defensesCybercity defenses with MLOpsAnomaly detectionRobustness tests against poisoningAdvanced poisoning defenses with ARTAdversarial trainingCreating a defense strategySummary
Chapter 5: Model Tampering with Trojan Horses and Model Reprogramming
Injecting backdoors using pickle serializationAttack scenarioDefenses and mitigationsInjecting Trojan horses with Keras Lambda layersAttack scenarioDefenses and mitigationsTrojan horses with custom layersAttack scenarioDefenses and mitigationsNeural payload injectionAttack scenarioDefenses and mitigationsAttacking edge AIAttack scenarioDefenses and mitigationsModel hijackingTrojan horse code injectionModel reprogrammingSummary
Chapter 6: Supply Chain Attacks and Adversarial AI
Traditional supply chain risks and AIRisks from outdated and vulnerable componentsRisks from AI’s dependency on live dataSecuring AI from vulnerable componentsEnhanced security – allow approved-only packagesClient configuration for private PyPI repositoriesAdditional private PyPI securityUse of SBOMsAI supply chain risksThe double-edged sword of transfer learningModel poisoningModel tamperingSecure model provenance and governance for pre-trained modelsMLOps and private model repositoriesData poisoningUsing data poisoning to affect sentiment analysisDefenses and mitigationsAI/ML SBOMsSummary

Part 3: Attacks on Deployed AI
Chapter 7: Evasion Attacks against Deployed AI
Fundamentals of evasion attacksImportance of understanding evasion attacksReconnaissance techniques for evasion attacksPerturbations and image evasion attack techniquesEvasion attack scenariosOne-step perturbation with FGSMBasic Iterative Method (BIM)Jacobian-based Saliency Map Attack (JSMA)Carlini and Wagner (C&W) attackProjected Gradient Descent (PGD)Adversarial patches – bridging digital and physical evasion techniquesNLP evasion attacks with BERT using TextAttackAttack scenario – sentiment analysisAttack exampleAttack scenario – natural language inferenceAttack exampleUniversal Adversarial Perturbations (UAPs)Attack scenarioAttack exampleBlack-box attacks with transferabilityAttack scenarioAttack exampleDefending against evasion attacksMitigation strategies overviewAdversarial trainingInput preprocessingModel hardening techniquesModel ensemblesCertified defensesSummary
Chapter 8: Privacy Attacks – Stealing Models
Understanding privacy attacksStealing models with model extraction attacksFunctionally equivalent extractionLearning-based model extraction attacksGenerative student-teacher extraction (distillation) attacksAttack example against our CIFAR-10 CNNDefenses and mitigationsPrevention measuresDetection measuresModel ownership identification and recoverySummary
Chapter 9: Privacy Attacks – Stealing Data
Understanding model inversion attacksTypes of model inversion attacksExploitation of model confidence scoresGAN-assisted model inversionExample model inversion attackUnderstanding inference attacksAttribute inference attacksMeta-classifiersPoisoning-assisted inferenceAttack scenariosMitigationsExample attribute inference attackMembership inference attacksStatistical thresholds for ML leaksLabel-only data transferring attackBlind membership inference attacksWhite box attacksAttack scenariosMitigationsExample membership inference attack using the ARTSummary
Chapter 10: Privacy-Preserving AI
Privacy-preserving ML and AISimple data anonymizationAdvanced anonymizationK-anonymityAnonymization and geolocation dataAnonymizing rich mediaDifferential privacy (DP)Federated learning (FL)Split learningAdvanced encryption options for privacy-preserving MLSecure multi-party computation (secure MPC)Homomorphic encryptionAdvanced ML encryption techniques in practiceApplying privacy-preserving ML techniquesSummary
Part 4: Generative AI and Adversarial Attacks
Chapter 11: Generative AI – A New Frontier
A brief introduction to generative AIA brief history of the evolution of generative AIGenerative AI technologiesUsing GANsDeveloping a GAN from scratchWGANs and custom loss functionsUsing pre-trained GANsPix2PixCycleGANPix2PixHDProgressive Growing of GANs (PGGAN)BigGANStarGAN v2StyleGAN seriesSummary
Chapter 12: Weaponizing GANs for Deepfakes and Adversarial Attacks
Use of GANs for deepfakes and deepfake detectionUsing StyleGAN to generate convincing fake imagesCreating simple deepfakes with GANs using existing imagesMaking direct changes to an existing imageUsing Pix2PixHD to synthesize imagesFake videos and animationsOther AI deepfake technologiesVoice deepfakesDeepfake detectionUsing GANs in cyberattacks and offensive securityEvading face verificationCompromising biometric authenticationPassword cracking with GANsMalware detection evasionGANs in cryptography and stenographyGenerating web attack payloads with GANsGenerating adversarial attack payloadsDefenses and mitigationsSecuring GANsGAN-assisted adversarial attacksDeepfakes, malicious content, and misinformationSummary
Chapter 13: LLM Foundations for Adversarial AI
A brief introduction to LLMsDeveloping AI applications with LLMsHello LLM with PythonHello LLM with LangChainBringing your own dataHow LLMs change Adversarial AISummary
Chapter 14: Adversarial Attacks with Prompts
Adversarial inputs and prompt injectionDirect prompt injectionPrompt overrideStyle injectionRole-playingImpersonationOther jailbreaking techniquesAutomated gradient-based prompt injectionRisks from bringing your own dataIndirect prompt injectionData exfiltration with prompt injectionPrivilege escalation with prompt injectionRCE with prompt injectionDefenses and mitigationsLLM platform defensesApplication-level defensesSummary
Chapter 15: Poisoning Attacks and LLMs
Poisoning embeddings in RAGAttack scenariosPoisoning during embedding generationDirect embeddings poisoningAdvanced embeddings poisoningQuery embeddings manipulationDefenses and mitigationsPoisoning attacks on fine-tuning LLMsIntroduction to fine-tuning LLMsFine-tuning poisoning attack scenariosFine-tuning attack vectorsPoisoning ChatGPT 3.5 with fine-tuningDefenses and mitigations against poisoning attacks in fine-tuningSummary
Chapter 16: Advanced Generative AI Scenarios
Supply-chain attacks in LLMsPublishing a poisoned LLM on Hugging FacePublishing a tampered LLM on Hugging FaceOther supply-chain risks for LLMsSupply-chain defenses and mitigationsPrivacy attacks and LLMsModel inversion and training data extraction attacks on LLMsInference attacks on LLMsModel cloning with LLMs using a secondary modelDefenses and mitigations for privacy attacksSummary
Part 5: Secure-by-Design AI and MLSecOps
Chapter 17: Secure by Design and Trustworthy AI
Secure by design AIBuilding our threat libraryTraditional cyber security threatsAdversarial AI attacksAdversarial AI attacks specific to Generative AISupply chain attacksIndustry AI threat taxonomiesAI threat taxonomy mappingNIST AI taxonomy mappingAI Exchange mappingMITRE ATLAS mappingThreat modeling for AIThreat modelling in actionExample AI solutionEnhanced FoodieAI threat modelRisk assessment and prioritizationApplying risk assessment to Enhanced FoodieAISecurity design and implementationTesting and verificationShifting left – embedding security into the AI life cycleLive operationsBeyond security – Trustworthy AISummary
Chapter 18: AI Security with MLSecOps
The MLSecOps imperativeToward an MLSecOps 2.0 frameworkMLSecOps orchestration optionsMLSecOps patternsBuilding a primary MLSecOPs platformMLSecOps in actionModel sourcing and validationIntegrating MLSecOps with LLMOpsAdvanced MLSecOps with SBOMsSummary
Chapter 19: Maturing AI Security
Enterprise security AI challengesFoundations of enterprise AI securityProtecting AI with enterprise securityOperational AI securityIterative enterprise securitySummary
Chapter 20: Unlock Your Book’s Exclusive Benefits
How to unlock these benefits in three easy stepsStep 1
Index
Why subscribe?
Other Books You May EnjoyPackt is searching for authors like youShare your thoughts

Content preview from Adversarial AI Attacks, Mitigations, and Defense Strategies

14 Adversarial Attacks with Prompts

In the previous chapter, we started looking at LLMs and how they change AI application development workflows and Adversarial AI. We looked at the evolution sparked by ChatGPT and the paradigm shift toward accessing external hosts via APIs rather than the model directly. With classic model development now done by specialist LLM developer organizations, the solution’s focus has shifted to sending inputs and outputs to the model using API calls.

As we’ll see, these calls use prompts, free-text inputs, mixing content, and instructions for the model and return similarly mixed content as output. These mixed inputs and outputs create new attack vectors for Adversarial AI, such as prompt injection, a term almost synonymous ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

AI Security and Responsible AI Practices

Publisher Resources

ISBN: 9781835087985

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

Adversarial AI Attacks, Mitigations, and Defense Strategies

by John Sotiropoulos

14

Adversarial Attacks with Prompts

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.