book

Federated Learning with Python

Name: Federated Learning with Python
ISBN: 9781803247106

by Kiyoshi Nakayama, PhD, George Jeno

October 2022

Intermediate to advanced

326 pages

8h 15m

English

Packt Publishing

Read now

Unlock full access

Federated Learning with Python
AcknowledgmentsContributorsAbout the authorsAbout the reviewer
Preface
Who this book is forWhat this book coversTo get the most out of this bookDownload the example code filesDownload the color imagesConventions usedGet in touchShare Your ThoughtsDownload a free PDF copy of this book
Part 1 Federated Learning – Conceptual Foundations
Chapter 1: Challenges in Big Data and Traditional AI
Understanding the nature of big dataDefinition of big dataBig data nowTriple-A mindset for big dataData privacy as a bottleneckRisks in handling private dataIncreased data protection regulationsFrom privacy by design to data minimalismImpacts of training data and model biasExpensive training of big dataModel bias and training dataModel drift and performance degradationHow models can stop workingContinuous monitoring – the price of letting causation goFL as the main solution for data problemsSummaryFurther reading
Chapter 2: What Is Federated Learning?
Understanding the current state of MLWhat is a model?ML – automating the model creation processDeep learningDistributed learning nature – toward scalable AIDistributed computingDistributed MLEdge inferenceEdge trainingUnderstanding FLDefining FLThe FL processFL system considerationsSecurity for FL systemsDecentralized FL and blockchainSummaryFurther reading
Chapter 3: Workings of the Federated Learning System
FL system architectureCluster aggregators Distributed agents Database serversIntermediate servers for low computational agent devicesUnderstanding the FL system flow – from initialization to continuous operationInitialization of the database, aggregator, and agentInitial model upload process by initial agentOverall FL cycle and process of the FL systemSynchronous and asynchronous FL The aggregator-side FL cycle and processThe agent-side local retraining cycle and processModel interpretation based on deviation from baseline outputsBasics of model aggregation What exactly does it mean to aggregate models?FedAvg – Federated averagingFurthering scalability with horizontal designHorizontal design with semi-global modelDistributed databaseAsynchronous agent participation in a multiple-aggregator scenarioSemi-global model synthesisSummaryFurther reading
Part 2 The Design and Implementation of the Federated Learning System
Chapter 4: Federated Learning Server Implementation with Python
Technical requirementsMain software components of the aggregator  and databaseAggregator-side codeslib/util codes Database-side codeToward the configuration of the aggregatorImplementing FL server-side functionalitiesImporting libraries for the FL serverDefining the FL Server class Initializing the FL server Registration function of agents The server for handling messages from local agents The global model synthesis routineFunctions to send the global models to the agents Functions to push the local and global models to the database Maintaining models for aggregation with the  state managerImporting the libraries of the state manager Defining the state manager classInitializing the state managerInitializing a global modelChecking the aggregation criteriaBuffering the local modelsClearing the saved modelsAdding agentsIncrementing the FL roundAggregating local modelsImporting the libraries for the aggregatorDefining and initializing the aggregator classDefining the aggregate_local_models function The FedAvg functionRunning the FL serverImplementing and running the database serverToward the configuration of the databaseDefining the database server efining the database with SQLiteRunning the database serverPotential enhancements to the FL serverRedesigning the database Automating the registry of an initial modelPerformance metrics for local and global modelsFine-tuned aggregationSummary
Chapter 5: Federated Learning Client-Side Implementation
Technical requirementsAn overview of FL client-side componentsDistributed agent-side codeConfiguration of an agentImplementing FL client-side main functionalitiesImporting libraries for an agentDefining the Client classInitializing the clientAgent participation in an FL cycleModel exchange synchronizationPush and polling implementationDesigning FL client librariesStarting FL client core threadsSaving global modelsManipulating client stateSending local models to aggregatorWaiting for global models from an aggregatorLocal ML engine integration into an FL systemImporting libraries for a local ML engineDefining the ML models, training, and test functionsIntegration of client libraries into your local ML engineAn example of integrating image classification  into an FL systemIntegration of client libraries into the IC exampleSummary
Chapter 6: Running the Federated Learning System and Analyzing the Results
Technical requirementsConfiguring and running the FL systemInstalling the FL environmentConfiguring the FL system with JSON files for each component Running the database and aggregator on the FL serverRunning a minimal example with the FL clientData and database foldersDatabases with SQLiteUnderstanding what happens when the minimal example runsRunning just one minimal agentRunning two minimal agentsRunning image classification and analyzing the resultsPreparing the CIFAR-10 datasetThe ML model used for FL with image classificationHow to run the image classification example with CNNEvaluation of running the image classification with CNN Running five agentsSummary

Chapter 7: Model Aggregation
Technical requirementsRevisiting aggregationUnderstanding FedAvgDataset distributionsComputational power distributionsProtecting against adversarial agentsModifying aggregation for non-ideal casesHandling heterogeneous computational powerAdversarial agentsNon-IID datasetsSummary
Part 3 Moving Toward the Production of Federated Learning Applications
Chapter 8: Introducing Existing Federated Learning Frameworks
Technical requirementsTensorFlow FederatedOpenFLIBM FLFlowerSTADLEIntroduction to FL frameworksFlowerTensorFlow Federated (TFF)OpenFLIBM FLSTADLEPySyftExample – the federated training of an NLP modelDefining the sentiment analysis modelCreating the data loaderTraining the modelAdopting an FL training approachIntegrating TensorFlow Federated for SST-2Integrating OpenFL for SST-2Integrating IBM FL for SST-2Integrating Flower for SST-2Integrating STADLE for SST-2Example – the federated training of an image classification model on non-IID dataSkewing the CIFAR-10 datasetIntegrating OpenFL for CIFAR-10Integrating IBM FL for CIFAR-10Integrating Flower for CIFAR-10Integrating STADLE for CIFAR-10Summary
Chapter 9: Case Studies with Key Use Cases of Federated Learning Applications
Applying FL to the healthcare sectorChallenges in healthcareMedical imagingDrug discoveryEHRsApplying FL to the financial sectorAnti-Money Laundering (AML)Proposed solutions to the existing AML approachDemo of FL in the AML spaceBenefits of FL for risk detection systemsFL meets edge computingEdge computing with IoT over 5GEdge FL example – object detectionMaking autonomous driving happen with FLApplying FL to roboticsMoving toward the Internet of IntelligenceIntroducing the IoFTUnderstanding the role of FL in Web 3.0Applying FL to distributed learning for big dataSummaryReferencesFurther reading
Chapter 10: Future Trends and Developments
Looking at future AI trendsThe limitation of centralized MLRevisiting the benefits of FLToward distributed learning for further privacy and training efficiencyOngoing research and developments in FLExploring various FL types and approachesUnderstanding enhanced distributed learning frameworks with FLJourneying on to collective intelligenceIntelligence-centric era with collective intelligenceInternet of IntelligenceCrowdsourced learning with FLSummaryFurther reading
Appendix: Exploring Internal Libraries
Technical requirementsOverview of the internal libraries for the FL systemstates.pycommunication_handler.pydata_struc.pyhelpers.pymessengers.pyEnumeration classes for implementing the FL systemImporting libraries to define the enumeration classesIDPrefix defining the FL system componentsClient state classesList of classes defining the types of ML models and messagesList of state classes defining message locationUnderstanding communication handler functionalitiesImporting libraries for the communication handlerFunctions of the communication handlerUnderstanding the data structure handler classImporting libraries for the data structure handlerThe LimitedDict classUnderstanding helper and supporting librariesImporting libraries for helper librariesFunctions of the helper libraryMessengers to generate communication payloadsImporting libraries for messengersFunctions of messengersSummary
Index
Why subscribe?
Other Books You May EnjoyPackt is searching for authors like youShare Your ThoughtsDownload a free PDF copy of this book

Content preview from Federated Learning with Python

packt.com

Subscribe to our online digital library for full access to over 7,000 books and videos, as well as industry leading tools to help you plan your personal development and advance your career. For more information, please visit our website.

Why subscribe?

Spend less time learning and more time coding with practical eBooks and Videos from over 4,000 industry professionals
Improve your learning with Skill Plans built especially for you
Get a free eBook or video every month
Fully searchable for easy access to vital information
Copy and paste, print, and bookmark content

Did you know that Packt offers eBook versions of every book published, ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

DevOps in Python: Infrastructure as Python

Publisher Resources

ISBN: 9781803247106

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

Federated Learning with Python

by Kiyoshi Nakayama, PhD, George Jeno

Why subscribe?

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.