book

AI, Machine Learning and Deep Learning

Name: AI, Machine Learning and Deep Learning
ISBN: 9781000878899

by Fei Hu, Xiali Hei

June 2023

Intermediate to advanced

346 pages

12h 39m

English

CRC Press

Read now

Unlock full access

Cover
Half Title
Title Page
Copyright Page
Table of Contents
Preface
About the Editors
Contributors
Part I Secure AI/ML Systems: Attack Models
1 Machine Learning Attack Models
1.1 Introduction1.2 Background1.2.1 Notation1.2.2 Support Vector Machines1.2.3 Neural Networks1.3 White-box adversarial attacks1.3.1 L-BGFS Attack1.3.2 Fast Gradient Sign Method1.3.3 Basic Iterative Method1.3.4 DeepFool1.3.5 Fast Adaptive Boundary Attack1.3.6 Carlini and Wagner’s Attack1.3.7 Shadow Attack1.3.8 Wasserstein Attack1.4 Black-box adversarial attacks1.4.1 Transfer Attack1.4.2 Score-based Black-box Attacks1.4.3 Decision-based Attack1.5 Data poisoning attacks1.5.1 Label Flipping Attacks1.5.2 Clean Label Data Poisoning Attack1.5.3 Backdoor Attack1.6 ConclusionsAcknowledgmentNoteReferences

2 Adversarial Machine Learning: A New Threat Paradigm for Next-generation Wireless Communications
2.1 Introduction2.1.1 Scope and Background2.2 Adversarial machine learning2.3 Challenges and gaps2.3.1 Development Environment2.3.2 Training and Test Datasets2.3.3 Repeatability, Hyperparameter Optimization, and Explainability2.3.4 Embedded Implementation2.4 Conclusions and recommendationsReferences
3 Threat of Adversarial Attacks to Deep Learning: A Survey
3.1 Introduction3.2 Categories of attacks3.2.1 White-box AttacksFGSM-based MethodJSMA-based Method3.2.2 Black-box AttacksMobility-based ApproachGradient Estimation-based Approach3.3 Attacks overview3.3.1 Attacks on Computer-Vision-Based Applications3.3.2 Attacks on Natural Language Processing Applications3.3.3 Attacks on Data Poisoning Applications3.4 Specific attacks in the real world3.4.1 Attacks on Natural Language Processing3.4.2 Attacks Using Data Poisoning3.5 Discussions and open issues3.6 ConclusionsReferences
4 Attack Models for Collaborative Deep Learning
4.1 Introduction4.2 Background4.2.1 Deep Learning (DL)Convolution Neural Network4.2.2 Collaborative Deep Learning (CDL)ArchitectureCollaborative Deep Learning Workflow4.2.3 Deep Learning Security and Collaborative Deep Learning Security4.3 Auror: an automated defense4.3.1 Problem Setting4.3.2 Threat ModelTargeted Poisoning Attacks4.3.3 AUROR Defense4.3.4 Evaluation4.4 A New cdl attack: gan attack4.4.1 Generative Adversarial Network (GAN)4.4.2 GAN AttackMain Protocol4.4.3 Experiment SetupsDatasetSystem ArchitectureHyperparameter Setup4.4.4 Evaluation4.5 Defend against gan attack in IoT4.5.1 Threat Model4.5.2 Defense System4.5.3 Main Protocols4.5.4 Evaluation4.6 ConclusionsAcknowledgmentReferences
5 Attacks on Deep Reinforcement Learning Systems: A Tutorial
5.1 Introduction5.2 Characterizing attacks on drl systems5.3 Adversarial attacks5.4 Policy induction attacks5.5 Conclusions and future directionsReferences
6 Trust and Security of Deep Reinforcement Learning
6.1 Introduction6.2 Deep reinforcement learning overview6.2.1 Markov Decision Process6.2.2 Value-based MethodsV-value FunctionQ-value FunctionAdvantage FunctionBellman Equation6.2.3 Policy-based Methods6.2.4 Actor–Critic Methods6.2.5 Deep Reinforcement Learning6.3 The most recent reviews6.3.1 Adversarial Attack on Machine Learning6.3.1.1 Evasion Attack6.3.1.2 Poisoning Attack6.3.2 Adversarial Attack on Deep Learning6.3.2.1 Evasion Attack6.3.2.2 Poisoning Attack6.3.3 Adversarial Deep Reinforcement Learning6.4 Attacks on drl systems6.4.1 Attacks on Environment6.4.2 Attacks on States6.4.3 Attacks on Policy Function6.4.4 Attacks on Reward Function6.5 Defenses against drl system attacks6.5.1 Adversarial Training6.5.2 Robust Learning6.5.3 Adversarial Detection6.6 Robust drl systems6.6.1 Secure Cloud Platform6.6.2 Robust DRL Modules6.7 A scenario of financial stability6.7.1 Automatic Algorithm Trading Systems6.8 Conclusion and future workReferences
7 IoT Threat Modeling Using Bayesian Networks
7.1 Background7.2 Topics of chapter7.3 Scope7.4 Cyber security in IoT networks7.4.1 Smart Home7.4.2 Attack Graphs7.5 Modeling with bayesian networks7.5.1 Graph Theory7.5.2 Probabilities and Distributions7.5.3 Bayesian Networks7.5.4 Parameter Learning7.5.5 Inference7.6 Model implementation7.6.1 Network Structure7.6.2 Attack SimulationSelection ProbabilitiesVulnerability Probabilities Based on CVSS ScoresAttack Simulation Algorithm7.6.3 Network Parametrization7.6.4 Results7.7 Conclusions and future workReferences
Part II Secure AI/ML Systems: Defenses
8 Survey of Machine Learning Defense Strategies
8.1 Introduction8.2 Security threats8.3 Honeypot defense8.4 Poisoned data defense8.5 Mixup inference against adversarial attacks8.6 Cyber-physical techniques8.7 Information fusion defense8.8 Conclusions and future directionsReferences
9 Defenses Against Deep Learning Attacks
9.1 Introduction9.2 Categories of defenses9.2.1 Modified Training or Modified InputData PreprocessingData Augmentation9.2.2 Modifying Networks ArchitectureNetwork DistillationModel Regularization9.2.3 Network Add-onDefense Against Universal PerturbationsMegNet Model9.4 Discussions and open issues9.5 ConclusionsReferences
10 Defensive Schemes for Cyber Security of Deep Reinforcement Learning
10.1 Introduction10.2 Background10.2.1 Model-free RL10.2.2 Deep Reinforcement Learning10.2.3 Security of DRL10.3 Certificated verification for adversarial examples10.3.1 Robustness Certification10.3.2 System Architecture10.3.3 Experimental Results10.4 Robustness on adversarial state observations10.4.1 State-adversarial DRL for Deterministic Policies: DDPG10.4.2 State-adversarial DRL for Q-Learning: DQN10.4.3 Experimental Results10.5 Conclusion and challengesAcknowledgmentReferences
11 Adversarial Attacks on Machine Learning Models in Cyber- Physical Systems
11.1 Introduction11.2 Support vector machine (svm) under evasion attacks11.2.1 Adversary Model11.2.2 Attack Scenarios11.2.3 Attack Strategy11.3 Svm under causality availability attack11.4 Adversarial label contamination on svm11.4.1 Random Label Flips11.4.2 Adversarial Label Flips11.5 ConclusionsReferences
12 Federated Learning and Blockchain: An Opportunity for Artificial Intelligence with Data Regulation
12.1 Introduction12.2 Data security and federated learning12.3 Federated learning context12.3.1 Type of Federation12.3.1.1 Model-centric Federated Learning12.3.1.2 Data-centric Federated Learning12.3.2 Techniques12.3.2.1 Horizontal Federated Learning12.3.2.2 Vertical Federated Learning12.4 Challenges12.4.1 Trade-off Between Efficiency and Privacy12.4.2 Communication Bottlenecks [1]12.4.3 Poisoning [2]12.5 Opportunities12.5.1 Leveraging Blockchain12.6 Use case: leveraging privacy, integrity, and availability for data-centric federated learning using a blockchain-based approach12.6.1 Results12.7 ConclusionReferences
Part III Using AI/ML Algorithms for Cyber Security
13 Using Machine Learning for Cyber Security: Overview
13.1 Introduction13.2 Is artificial intelligence enough to stop cyber crime?13.3 Corporations’ use of machine learning to strengthen their cyber security systems13.4 Cyber attack/cyber security threats and attacks13.4.1 Malware13.4.2 Data Breach13.4.3 Structured Query Language Injection (SQL-i)13.4.4 Cross-site Scripting (XSS)13.4.5 Denial-of-service (DOS) Attack13.4.6 Insider Threats13.4.7 Birthday Attack13.4.8 Network Intrusions13.4.9 Impersonation Attacks13.4.10 DDoS Attacks Detection on Online Systems13.5 Different machine learning techniques in cyber security13.5.1 Support Vector Machine (SVM)13.5.2 K-nearest Neighbor (KNN)13.5.3 Naïve Bayes13.5.4 Decision Tree13.5.5 Random Forest (RF)13.5.6 Multilayer Perceptron (MLP)13.6 Application of machine learning13.6.1 ML in Aviation Industry13.6.2 Cyber ML Under Cyber Security Monitoring13.6.3 Battery Energy Storage System (BESS) Cyber Attack Mitigation13.6.4 Energy-based Cyber Attack Detection in Large-Scale Smart Grids13.6.5 IDS for Internet of Vehicles (IoV)13.7 Deep learning techniques in cyber security13.7.1 Deep Auto-encoder13.7.2 Convolutional Neural Networks (CNN)13.7.3 Recurrent Neural Networks (RNNs)13.7.4 Deep Neural Networks (DNNs)13.7.5 Generative Adversarial Networks (GANs)13.7.6 Restricted Boltzmann Machine (RBM)13.7.7 Deep Belief Network (DBN)13.8 Applications of deep learning in cyber security13.8.1 Keystroke Analysis13.8.2 Secure Communication in IoT13.8.3 Botnet Detection13.8.4 Intrusion Detection and Prevention Systems (IDS/IPS)13.8.5 Malware Detection in Android13.8.6 Cyber Security Datasets13.8.7 Evaluation Metrics13.9 ConclusionReferences
14 Performance of Machine Learning and Big Data Analytics Paradigms in Cyber Security
14.1 Introduction14.1.1 Background on Cyber Security and Machine Learning14.1.2 Background Perspectives to Big Data Analytics and Cyber Security14.1.3 Supervised Learning Algorithms14.1.4 Statement of the Problem14.1.5 Purpose of Study14.1.6 Research Objectives14.1.7 Research Questions14.2 Literature review14.2.1 Overview14.2.2 Classical Machine Learning (CML)14.2.2.1 Logistic Regression (LR)14.2.2.2 Naïve Bayes (NB)14.2.2.3 Decision Tree (DT)14.2.2.4 K-nearest Neighbor (KNN)14.2.2.5 AdaBoost (AB)14.2.2.6 Random Forest (RF)14.2.2.7 Support Vector Machine (SVM)14.2.3 Modern Machine Learning14.2.3.1 Deep Neural Network (DNN)14.2.3.2 Future of AI in the Fight against Cyber Crimes14.2.4 Big Data Analytics and Cyber Security14.2.4.1 Big Data Analytics Issues14.2.4.2 Independent Variable: Big Data Analytics14.2.4.3 Intermediating Variables14.2.4.4 Conceptual Framework14.2.4.5 Theoretical Framework14.2.4.6 Big Data Analytics Application to Cyber Security14.2.4.7 Big Data Analytics and Cyber Security Limitations14.2.4.8 Limitations14.2.5 Advances in Cloud Computing14.2.5.1 Explaining Cloud Computing and How It Has Evolved to Date14.2.6 Cloud Characteristics14.2.7 Cloud Computing Service Models14.2.7.1 Software as a Service (SaaS)14.2.7.2 Platform as a Service (PaaS)14.2.7.3 Infrastructure as a Service (IaaS)14.2.8 Cloud Deployment Models14.2.8.1 Private Cloud14.2.8.2 Public Cloud14.2.8.3 Hybrid Cloud14.2.8.4 Community Cloud14.2.8.5 Advantages and Disadvantages of Cloud Computing14.2.8.6 Six Main Characteristics of Cloud Computing and How They Are Leveraged14.2.8.7 Some Advantages of Network Function Virtualization14.2.8.8 Virtualization and Containerization Compared and Contrasted14.3 Research methodology14.3.1 Presentation of the Methodology14.3.1.1 Research Approach and Philosophy14.3.1.2 Research Design and Methods14.3.2 Population and Sampling14.3.2.1 Population14.3.2.2 Sample14.3.3 Sources and Types of Data14.3.4 Model for Analysis14.3.4.1 Big Data14.3.4.2 Big Data Analytics14.3.4.3 Insights for Action14.3.4.4 Predictive Analytics14.3.5 Validity and Reliability14.3.6 Summary of Research Methodology14.3.7 Possible Outcomes14.4 Analysis and research outcomes14.4.1 Overview14.4.2 Support Vector Machine14.4.3 KNN Algorithm14.4.4 Multilinear Discriminant Analysis (LDA)14.4.5 Random Forest Classifier14.4.6 Variable Importance14.4.7 Model Results14.4.8 Classification and Regression Trees (CART)14.4.9 Support Vector Machine14.4.10 Linear Discriminant Algorithm14.4.11 K-Nearest Neighbor14.4.12 Random Forest14.4.13 Challenges and Future Direction14.4.13.1 Model 1: Experimental/Prototype Model14.4.13.2 Model 2: Cloud Computing/Outsourcing14.4.13.3 Application of Big Data Analytics Models in Cyber Security14.4.13.4 Summary of Analysis14.5 ConclusionReferences
15 Using ML and DL Algorithms for Intrusion Detection in the Industrial Internet of Things
15.1 Introduction15.2 Ids applications15.2.1 Random Forest Classifier15.2.2 Pearson Correlation Coefficient15.2.3 Related Works15.3 Use of ml and dl algorithms in iiot applications15.4 Practical application of ml algorithms in iiot15.4.1 Results15.5 ConclusionReferences
Part IV Applications
16 On Detecting Interest Flooding Attacks in Named Data Networking (NDN)–based IoT Searches
16.1 Introduction16.2 Preliminaries16.2.1 Named Data Networking (NDN)16.2.2 Internet of Things Search Engine (IoTSE)16.2.3 Machine Learning (ML)16.3 Machine learning assisted for ndn-based ifa detection in iotse16.3.1 Attack Model16.3.2 Attack Scale16.3.3 Attack Scenarios16.3.4 Machine Learning (ML) Detection Models16.4 Performance evaluation16.4.1 Methodology16.4.2 IFA Performance16.4.2.1 Simple Tree Topology (Small Scale)16.4.2.2 Rocketfuel ISP like Topology (Large Scale)16.4.3 Data Processing for Detection16.4.4 Detection Results16.4.4.1 ML Detection Performance in Simple Tree Topology16.4.4.2 ML Detection in Rocketful ISP Topology16.5 Discussion16.6 Related works16.7 Final remarksAcknowledgmentReferences
17 Attack on Fraud Detection Systems in Online Banking Using Generative Adversarial Networks
17.1 Introduction17.1.1 Problem of Fraud Detection in Banking17.1.2 Fraud Detection and Prevention System17.2 Experiment description17.2.1 Research Goal17.2.2 Empirical Data17.2.3 Attack Scenario17.3 Generator and discrimination model17.3.1 Model Construction17.3.1.1 Imitation Fraud Detection System Model17.3.1.2 Generator Models17.3.2 Evaluation of Models17.4 Final conclusions and recommendationsNotesReferences
18 Artificial Intelligence-assisted Security Analysis of Smart Healthcare Systems
18.1 Introduction18.2 Smart healthcare system (shs)18.2.1 Formal Modeling of SHS18.2.2 Machine Learning (ML)–based Patient Status Classification Module (PSCM) in SHS18.2.2.1 Decision Tree (DT)18.2.2.2 Logistic Regression (LR)18.2.2.3 Neural Network (NN)18.2.3 Hyperparameter Optimization of PSCM in SHS18.2.3.1 Whale Optimization (WO)18.2.3.2 Grey Wolf Optimization (GWO)18.2.3.3 Firefly Optimization (FO)18.2.3.4 Evaluation Results18.3 Formal attack modeling of shs18.3.1 Attacks in SHS18.3.2 Attacker’s Knowledge18.3.3 Attacker’s Capability18.3.4 Attacker’s Accessibility18.3.5 Attacker’s Goal18.4 Anomaly detection models (adms) in shs18.4.1 ML-based Anomaly Detection Model (ADM) in SHS18.4.1.1 Density-based Spatial Clustering of Applications with Noise (DBSCAN)18.4.1.2 K-means18.4.1.3 One-class SVM (OCSVM)18.4.1.4 Autoencoder (AE)18.4.2 Ensemble-based ADMs in SHS18.4.2.1 Data Collection and Preprocessing18.4.2.2 Model Training18.4.2.3 Threshold Calculation18.4.2.4 Anomaly Detection18.4.2.5 Example Case Studies18.4.2.6 Evaluation Result18.4.2.7 Hyperparameter Optimization of ADMs in SHS18.5 Formal attack analysis of smart healthcare systems18.5.1 Example Case Studies18.5.2 Performance with Respect to Attacker Capability18.5.3 Frequency of Sensors in the Attack Vectors18.5.4 Scalability Analysis18.6 Resiliency analysis of smart healthcare system18.7 Conclusion and future worksReferences
19 A User-centric Focus for Detecting Phishing Emails
19.1 Introduction19.2 Background and related work19.2.1 Behavioral Models Related to Phishing Susceptibility19.2.2 User-centric Antiphishing Measures19.2.3 Technical Antiphishing Measures19.2.4 Research Gap19.3 The dataset19.4 Understanding the decision behavior of machine learning models19.4.1 Interpreter for Machine Learning Algorithms19.4.2 Local Interpretable Model-Agnostic Explanations (LIME)19.4.3 Anchor Explanations19.4.3.1 Share of Emails in the Data for Which the Rule Holds19.5 Designing the artifact19.5.1 Background19.5.2 Identifying Suspected Phishing Attempts19.5.3 Cues in Phishing Emails19.5.4 Extracting Cues19.5.5 Examples of the Application of XAI for Extracting Cues and Phrases19.6 Conclusion and future works19.6.1 Completion of the ArtifactNotesReferences

Content preview from AI, Machine Learning and Deep Learning

5Attacks on Deep Reinforcement Learning Systems: A Tutorial

Joseph Layton¹ and Fei Hu¹

¹Electrical and Computer Engineering, University of Alabama, USA

DOI: 10.1201/9781003187158-6

Contents

5.1 INTRODUCTION

Deep reinforcement learning (DRL) systems combine the techniques of reinforcement learning (reward maximization) with the artificial neural networks of deep learning (DL) [1]. This combination allows a well trained model to operate on much more complex data sets than a typical reinforcement learning system. For example, input data from a camera of sensor stream is high dimensional ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Publisher Resources

ISBN: 9781000878899

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

AI, Machine Learning and Deep Learning

by Fei Hu, Xiali Hei

5Attacks on Deep Reinforcement Learning Systems: A Tutorial

5.1 INTRODUCTION

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.