book

Machine Learning Engineering with Python - Second Edition

Name: Machine Learning Engineering with Python - Second Edition
Author: Andrew P. McMahon
ISBN: 9781837631964

by Andrew P. McMahon

August 2023

Intermediate to advanced

462 pages

11h 20m

English

Packt Publishing

Read now

Unlock full access

Preface
Who this book is forWhat this book coversTo get the most out of this bookGet in touch
Introduction to ML Engineering
Technical requirementsDefining a taxonomy of data disciplinesData scientistML engineerML operations engineerData engineerWorking as an effective teamML engineering in the real worldWhat does an ML solution look like?Why Python?High-level ML system designExample 1: Batch anomaly detection serviceExample 2: Forecasting APIExample 3: Classification pipelineSummary
The Machine Learning Development Process
Technical requirementsSetting up our toolsSetting up an AWS accountConcept to solution in four stepsComparing this to CRISP-DMDiscoverUsing user storiesPlayDevelopSelecting a software development methodologyPackage management (conda and pip)PoetryCode version controlGit strategiesModel version controlDeployKnowing your deployment optionsUnderstanding DevOps and MLOpsBuilding our first CI/CD example with GitHub ActionsContinuous model performance testingContinuous model trainingSummary
From Model to Model Factory
Technical requirementsDefining the model factoryLearning about learningDefining the targetCutting your lossesPreparing the dataEngineering features for machine learningEngineering categorical featuresEngineering numerical featuresDesigning your training systemTraining system design optionsTrain-runTrain-persistRetraining requiredDetecting data driftDetecting concept driftSetting the limitsDiagnosing the driftRemediating the driftOther tools for monitoringAutomating trainingHierarchies of automationOptimizing hyperparametersHyperoptOptunaAutoMLauto-sklearnAutoKerasPersisting your modelsBuilding the model factory with pipelinesScikit-learn pipelinesSpark ML pipelinesSummary
Packaging Up
Technical requirementsWriting good PythonRecapping the basicsTips and tricksAdhering to standardsWriting good PySparkChoosing a styleObject-oriented programmingFunctional programmingPackaging your codeWhy package?Selecting use cases for packagingDesigning your packageBuilding your packageManaging your environment with MakefilesGetting all poetic with PoetryTesting, logging, securing, and error handlingTestingSecuring your solutionsAnalyzing your own code for security issuesAnalyzing dependencies for security issuesLoggingError handlingNot reinventing the wheelSummary
Deployment Patterns and Tools
Technical requirementsArchitecting systemsBuilding with principlesExploring some standard ML patternsSwimming in data lakesMicroservicesEvent-based designsBatchingContainerizingHosting your own microservice on AWSPushing to ECRHosting on ECSBuilding general pipelines with AirflowAirflowAirflow on AWSRevisiting CI/CD for AirflowBuilding advanced ML pipelinesFinding your ZenMLGoing with the KubeflowSelecting your deployment strategySummary
Scaling Up
Technical requirementsScaling with SparkSpark tips and tricksSpark on the cloudAWS EMR exampleSpinning up serverless infrastructureContainerizing at scale with KubernetesScaling with RayGetting started with Ray for MLScaling your compute for RayScaling your serving layer with RayDesigning systems at scaleSummary
Deep Learning, Generative AI, and LLMOps
Going deep with deep learningGetting started with PyTorchScaling and taking deep learning into productionFine-tuning and transfer learningLiving it large with LLMsUnderstanding LLMsConsuming LLMs via APICoding with LLMsBuilding the future with LLMOpsValidating LLMsPromptOpsSummary
Building an Example ML Microservice
Technical requirementsUnderstanding the forecasting problemDesigning our forecasting serviceSelecting the toolsTraining at scaleServing the models with FastAPIResponse and request schemasManaging models in your microservicePulling it all togetherContainerizing and deploying to KubernetesContainerizing the applicationScaling up with KubernetesDeployment strategiesSummary
Building an Extract, Transform, Machine Learning Use Case
Technical requirementsUnderstanding the batch processing problemDesigning an ETML solutionSelecting the toolsInterfaces and storageScaling of modelsScheduling of ETML pipelinesExecuting the buildBuilding an ETML pipeline with advanced Airflow featuresSummary

Other Books You May Enjoy
Index

Content preview from Machine Learning Engineering with Python - Second Edition

4 Packaging Up

In previous chapters, we introduced a lot of the tools and techniques you will need to use to successfully build working machine learning (ML) products. We also introduced a lot of example pieces of code that helped us to understand how to implement these tools and techniques. So far, this has all been about what we need to program, but this chapter will focus on how to program. In particular, we will introduce and work with a lot of the techniques, methodologies, and standards that are prevalent in the wider Python software development community and apply them to ML use cases. The conversation will be centered around the concept of developing user-defined libraries and packages, reusable pieces of code that you can use to deploy ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Machine Learning Engineering with Python

Publisher Resources

ISBN: 9781837631964

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

Machine Learning Engineering with Python - Second Edition

by Andrew P. McMahon

4

Packaging Up

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.