book

Machine Learning and Security

by Clarence Chio, David Freeman

February 2018

Intermediate to advanced

383 pages

11h 30m

English

O'Reilly Media, Inc.

Book available

Read now

Unlock full access

What’s In This Book?Who Is This Book For?Conventions Used in This BookUsing Code ExamplesO’Reilly SafariHow to Contact UsAcknowledgments
Cyber Threat LandscapeThe Cyber Attacker’s EconomyA Marketplace for Hacking SkillsIndirect MonetizationThe UpshotWhat Is Machine Learning?What Machine Learning Is NotAdversaries Using Machine LearningReal-World Uses of Machine Learning in SecuritySpam Fighting: An Iterative ApproachLimitations of Machine Learning in Security
Machine Learning: Problems and ApproachesMachine Learning in Practice: A Worked ExampleTraining Algorithms to LearnModel FamiliesLoss FunctionsOptimizationSupervised Classification AlgorithmsLogistic RegressionDecision TreesDecision ForestsSupport Vector MachinesNaive Bayesk-Nearest NeighborsNeural NetworksPractical Considerations in ClassificationSelecting a Model FamilyTraining Data ConstructionFeature SelectionOverfitting and UnderfittingChoosing Thresholds and Comparing ModelsClusteringClustering AlgorithmsEvaluating Clustering ResultsConclusion
When to Use Anomaly Detection Versus Supervised LearningIntrusion Detection with HeuristicsData-Driven MethodsFeature Engineering for Anomaly DetectionHost Intrusion DetectionNetwork Intrusion DetectionWeb Application Intrusion DetectionIn SummaryAnomaly Detection with Data and AlgorithmsForecasting (Supervised Machine Learning)Statistical MetricsGoodness-of-FitUnsupervised Machine Learning AlgorithmsDensity-Based MethodsIn SummaryChallenges of Using Machine Learning in Anomaly DetectionResponse and MitigationPractical System Design ConcernsOptimizing for ExplainabilityMaintainability of Anomaly Detection SystemsIntegrating Human FeedbackMitigating Adversarial EffectsConclusion
Understanding MalwareDefining Malware ClassificationMalware: Behind the ScenesFeature GenerationData CollectionGenerating FeaturesFeature SelectionFrom Features to ClassificationHow to Get Malware Samples and LabelsConclusion
Theory of Network DefenseAccess Control and AuthenticationIntrusion DetectionDetecting In-Network AttackersData-Centric SecurityHoneypotsSummaryMachine Learning and Network SecurityFrom Captures to FeaturesThreats in the NetworkBotnets and YouBuilding a Predictive Model to Classify Network AttacksExploring the DataData PreparationClassificationSupervised LearningSemi-Supervised LearningUnsupervised LearningAdvanced EnsemblingConclusion
Monetizing the Consumer WebTypes of Abuse and the Data That Can Stop ThemAuthentication and Account TakeoverAccount CreationFinancial FraudBot ActivitySupervised Learning for Abuse ProblemsLabeling DataCold Start Versus Warm StartFalse Positives and False NegativesMultiple ResponsesLarge AttacksClustering AbuseExample: Clustering Spam DomainsGenerating ClustersScoring ClustersFurther Directions in ClusteringConclusion
Defining Machine Learning System Maturity and ScalabilityWhat’s Important for Security Machine Learning Systems?Data QualityProblem: Bias in DatasetsProblem: Label InaccuracySolutions: Data QualityProblem: Missing DataSolutions: Missing DataModel QualityProblem: Hyperparameter OptimizationSolutions: Hyperparameter OptimizationFeature: Feedback Loops, A/B Testing of ModelsFeature: Repeatable and Explainable ResultsPerformanceGoal: Low Latency, High ScalabilityPerformance OptimizationHorizontal Scaling with Distributed Computing FrameworksUsing Cloud ServicesMaintainabilityProblem: Checkpointing, Versioning, and Deploying ModelsGoal: Graceful DegradationGoal: Easily Tunable and ConfigurableMonitoring and AlertingSecurity and ReliabilityFeature: Robustness in Adversarial ContextsFeature: Data Privacy Safeguards and GuaranteesFeedback and UsabilityConclusion
TerminologyThe Importance of Adversarial MLSecurity Vulnerabilities in Machine Learning AlgorithmsAttack TransferabilityAttack Technique: Model PoisoningExample: Binary Classifier Poisoning AttackAttacker KnowledgeDefense Against Poisoning AttacksAttack Technique: Evasion AttackExample: Binary Classifier Evasion AttackDefense Against Evasion AttacksConclusion
More About MetricsSize of Logistic Regression ModelsImplementing the Logistic Regression Cost FunctionMinimizing the Cost Function

Security Intelligence FeedsGeolocation

Content preview from Machine Learning and Security

Chapter 5. Network Traffic Analysis

The most likely way that attackers will gain entry to your infrastructure is through the network. Network security is the broad practice of protecting computer networks and network-accessible endpoints from malice, misuse, and denial.¹ Firewalls are perhaps the best-known network defense systems, enforcing access policies and filtering unauthorized traffic between artifacts in the network. However, network defense is about more than just firewalls.

In this chapter, we look at techniques for classifying network traffic. We begin by building a model of network defense upon which we will base our discussions. Then, we dive into selected verticals within network security that have benefited from developments in artificial intelligence and machine learning. In the second part of this chapter, we work through an example of using machine learning to find patterns and discover correlations in network data. Using data science as an investigation tool, we discover how to apply classification on complex datasets to uncover attackers within the network.

Our discussion of network security is limited to packet-based information transmission. In packet-based transmission, a data stream is segmented into smaller units, each of which contains some metadata about the transmission origin, destination, and content. Each packet is transmitted over the network layer and formatted in an appropriate protocol by the transport layer, with the reconstruction of the information ...