book

Machine Learning for Streaming Data with Python

by Joos Korstanje

July 2022

Beginner to intermediate

258 pages

4h 57m

English

Packt Publishing

Read now

Unlock full access

ContributorsAbout the authorAbout the reviewer
Who this book is forWhat this book coversTo get the most out of this bookDownload the example code filesDownload the color imagesConventions usedGet in touchShare Your Thoughts
Technical requirementsSetting up a Python environmentA short history of data scienceWorking with streaming dataStreaming data versus batch dataAdvantages of streaming dataExamples of successful implementation of streaming analyticsChallenges of streaming dataHow to get started with streaming dataCommon use cases for streaming dataStreaming versus big dataReal-time data formats and importing an example dataset in PythonSummaryFurther reading
Technical requirementsPython environmentDefining your analytics as a functionUnderstanding microservices architectureCommunicating between services through APIsDemystifying the HTTP protocolThe GET requestThe POST requestJSON format for communication between systemsRESTful APIsBuilding a simple API on AWSAPI Gateway in AWSLambda in AWSData-generating process on a local machineImplementing the exampleMore architectural considerationsOther AWS services and other services in general that have the same functionalityBig data tools for real time streamingCalling a big data environment in real timeSummaryFurther reading
Technical requirementsPython environmentDescriptive statistics on streaming dataWhy are descriptive statistics different on streaming data?Introduction to sampling theoryComparing population and samplePopulation parameters and sample statisticsSampling distributionSample size calculations and confidence levelRolling descriptive statistics from streamingExponential weightTracking convergence as an additional KPIOverview of the main descriptive statisticsThe meanThe medianThe modeStandard deviationVarianceQuartiles and interquartile rangeCorrelationsReal-time visualizationsOpening the dashboardComparing Plotly's Dash and other real-time visualization toolsBuilding basic alerting systemsAlerting systems on extreme valuesAlerting systems on process stability (mean and median)Alerting systems on constant variability (std and variance)Basic alerting systems using statistical process controlSummaryFurther reading
Technical requirementsPython environmentWhat is online machine learning?How is online learning different from regular learning?Advantages of online learningChallenges of online learningTypes of online learningUsing River for online learningTraining an online model with RiverImproving the model evaluationBuilding a multiclass classifier using one-vs-restSummaryFurther reading
Technical requirementsPython environmentDefining anomaly detectionAre outliers a problem?Exploring use cases of anomaly detectionFraud detection in financial institutionsAnomaly detection on your log dataFault detection in manufacturing and production linesHacking detection in computer networks (cyber security)Medical risks in health dataPredictive maintenance and sensor dataComparing anomaly detection and imbalanced classificationThe problem of imbalanced dataThe F1 scoreSMOTE oversamplingAnomaly detection versus classificationAlgorithms for detecting anomalies in RiverThe use of thresholders in River anomaly detectionAnomaly detection algorithm 1 – One-Class SVMAnomaly detection algorithm 2 – Half-Space-TreesGoing further with anomaly detectionSummaryFurther reading
Technical requirementsPython environmentDefining classificationIdentifying use cases of classificationUse case 1 – email spam classificationUse case 2 – face detection in phone cameraUse case 3 – online marketing ad selectionOverview of classification algorithms in RiverClassification algorithm 1 – LogisticRegressionClassification algorithm 2 – PerceptronClassification algorithm 3 – AdaptiveRandomForestClassifierClassification algorithm 4 – ALMAClassifierClassification algorithm 5 – PAClassifierEvaluating benchmark resultsSummaryFurther reading

Technical requirementsPython environmentDefining regressionUse cases of regressionUse case 1 – ForecastingUse case 2 – Predicting the number of faulty products in manufacturingOverview of regression algorithms in RiverRegression algorithm 1 – LinearRegressionRegression algorithm 2 – HoeffdingAdaptiveTreeRegressorRegression algorithm 3 – SGTRegressorRegression algorithm 4 – SRPRegressorSummaryFurther reading
Technical requirementsPython environmentDefining reinforcement learningComparing online and offline reinforcement learningA more detailed overview of feedback loops in reinforcement learningThe main steps of a reinforcement learning modelMaking the decisionsUpdating the decision rulesExploring Q-learningThe goal of Q-learningParameters of the Q-learning algorithmDeep Q-learningUsing reinforcement learning for streaming dataUse cases of reinforcement learningUse case one – trading systemUse case two – social network ranking systemUse case three – a self-driving carUse case four – chatbotsUse case five – learning gamesImplementing reinforcement learning in PythonSummaryFurther reading
Technical requirementsPython environmentDefining driftThree types of driftIntroducing model explicabilityMeasuring driftMeasuring data driftMeasuring concept driftMeasuring drift in PythonA basic intuitive approach to measuring driftMeasuring drift with robust toolsCounteracting driftOffline learning with retraining strategies against driftOnline learning against driftSummaryFurther reading
Technical requirementsPython environmentChallenges of data preparation with streaming dataScaling data for streamingIntroducing scalingAdapting scaling to a streaming contextTransforming features in a streaming contextIntroducing PCAMathematical definition of PCARegular PCA in PythonIncremental PCA for streamingSummaryFurther reading
Technical requirementsPython environmentIntroducing catastrophic forgettingCatastrophic forgetting in online modelsDetecting catastrophic forgettingUsing Python to detect catastrophic forgettingModel explicability versus catastrophic forgettingExplaining models using linear coefficientsExplaining models using dendrogramsExplaining models using variable importanceSummaryFurther reading
Going furtherSummary
Other Books You May EnjoyPackt is searching for authors like youShare Your Thoughts

Content preview from Machine Learning for Streaming Data with Python

Chapter 4: Online Learning with River

In this and the coming three chapters, you will learn how to work with a library for online machine learning called River. Online machine learning is a part of machine learning in which models are designed in such a way that they can update their learned model on the reception of any new data point.

Online machine learning is the opposite of offline machine learning, which is the regular machine learning that you are probably already aware of. In general, in machine learning, a model will try to learn a mathematical rule that can perform a certain task. This task is learned on the basis of a number of data points. The mathematics behind these tasks is based on statistics and algorithmics.

In this chapter, ...