book

AI Crash Course

Name: AI Crash Course
Author: Hadelin de Ponteves
ISBN: 9781838645359

by Hadelin de Ponteves

November 2019

Intermediate to advanced

360 pages

8h 31m

English

Packt Publishing

Read now

Unlock full access

Why subscribe?
ContributorsAbout the authorAbout the reviewers
Preface
Who this book is forWhat this book coversTo get the most out of this bookDownload the example code filesDownload the color imagesConventions usedGet in touchReviews
Welcome to the Robot World
Beginning the AI journeyFour different AI modelsThe models in practiceFundamentalsThompson SamplingQ-learningDeep Q-learningDeep convolutional Q-learningWhere can learning AI take you?EnergyHealthcareTransport and logisticsEducationSecurityEmploymentSmart homes and robotsEntertainment and happinessEnvironmentEconomy, business, and financeSummary
Discover Your AI Toolkit
The GitHub pageColaboratorySummary
Python Fundamentals – Learn How to Code in Python
Displaying textExerciseVariables and operationsExerciseLists and arraysExerciseif statements and conditionsExercisefor and while loopsExerciseFunctionsExerciseClasses and objectsExerciseSummary
AI Foundation Techniques
What is Reinforcement Learning?The five principles of Reinforcement LearningPrinciple #1 – The input and output systemPrinciple #2 – The rewardPrinciple #3 – The AI environmentPrinciple #4 – The Markov decision processPrinciple #5 – Training and inferenceTraining modeInference modeSummary
Your First AI Model – Beware the Bandits!
The multi-armed bandit problemThe Thompson Sampling modelCoding the modelUnderstanding the modelWhat is a distribution?Tackling the MABPThe Thompson Sampling strategy in three stepsThe final touch of shaping your Thompson Sampling intuitionThompson Sampling against the standard modelSummary
AI for Sales and Advertising – Sell like the Wolf of AI Street
Problem to solveBuilding the environment inside a simulationRunning the simulationRecapAI solution and intuition refresherAI solutionIntuitionImplementationThompson Sampling vs. Random SelectionPerformance measureLet's start codingThe final resultSummary
Welcome to Q-Learning
The MazeBeginningsBuilding the environmentThe statesThe actionsThe rewardsBuilding the AIThe Q-valueThe temporal differenceThe Bellman equationReinforcement intuitionThe whole Q-learning processTraining modeInference modeSummary
AI for Logistics – Robots in a Warehouse
Building the environmentThe statesThe actionsThe rewardsAI solution refresherInitialization (first iteration)Next iterationsImplementationPart 1 – Building the environmentPart 2 – Building the AI Solution with Q-learningPart 3 – Going into productionImprovement 1 – Automating reward attributionImprovement 2 – Adding an intermediate goalSummary

Going Pro with Artificial Brains – Deep Q-Learning
Predicting house pricesUploading the datasetImporting librariesExcluding variablesData preparationScaling dataBuilding the neural networkTraining the neural networkDisplaying resultsDeep learning theoryThe neuronBiological neuronsArtificial neuronsThe activation functionThe threshold activation functionThe sigmoid activation functionThe rectifier activation functionHow do neural networks work?How do neural networks learn?Forward-propagation and back-propagationGradient DescentBatch gradient descentStochastic gradient descentMini-batch gradient descentDeep Q-learningThe Softmax methodDeep Q-learning recapExperience replayThe whole deep Q-learning algorithmSummary
AI for Autonomous Vehicles – Build a Self-Driving Car
Building the environmentDefining the goalSetting the parametersThe input statesThe output actionsThe rewardsAI solution refresherImplementationStep 1 – Importing the librariesStep 2 – Creating the architecture of the neural networkStep 3 – Implementing experience replayStep 4 – Implementing deep Q-learningThe demoInstalling AnacondaCreating a virtual environment with Python 3.6Installing PyTorchInstalling KivySummary
AI for Business – Minimize Costs with Deep Q-Learning
Problem to solveBuilding the environmentParameters and variables of the server environmentAssumptions of the server environmentAssumption 1 – We can approximate the server temperatureAssumption 2 – We can approximate the energy costsSimulationOverall functioningDefining the statesDefining the actionsDefining the rewardsFinal simulation exampleAI solutionThe brainImplementationStep 1 – Building the environmentStep 2 – Building the brainWithout dropoutWith dropoutStep 3 – Implementing the deep reinforcement learning algorithmStep 4: Training the AINo early stoppingEarly stoppingStep 5 – Testing the AIThe demoRecap – The general AI framework/BlueprintSummary
Deep Convolutional Q-Learning
What are CNNs used for?How do CNNs work?Step 1 – ConvolutionStep 2 – Max poolingStep 3 – FlatteningStep 4 – Full connectionDeep convolutional Q-learningSummary
AI for Games – Become the Master at Snake
Problem to solveBuilding the environmentDefining the statesDefining the actionsDefining the rewardsAI solutionThe brainThe experience replay memoryImplementationStep 1 – Building the environmentStep 2 – Building the brainStep 3 – Building the experience replay memoryStep 4 – Training the AIStep 5 – Testing the AIThe demoInstallationThe resultsSummary
Recap and Conclusion
Recap – The general AI framework/blueprintExploring what's next for you in AIPractice, practice, and practiceNetworkingNever stop learning
Other Books You May Enjoy
Leave a review - let other readers know what you think
Index

Content preview from AI Crash Course

7 Welcome to Q-Learning

Ladies and gentlemen, things are about to get even more interesting than before. The next model we are about to tackle is at the heart of many AIs built today; robots, autonomous vehicles, and even AI players of video games. They all use Q-learning at the core of their model. Some of them even combine Q-learning with deep learning, making a highly advanced version of Q-learning called deep Q-learning, which we will cover in Chapter 9, Going Pro with Artificial Brains – Deep Q-Learning.

All of the AI fundamentals still apply to Q-learning, as follows:

Q-learning is a Reinforcement Learning model.
Q-learning works on the inputs (states) and outputs (actions) principle.
Q-learning works on a predefined environment, including ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Hands-On Artificial Intelligence for Beginners

Publisher Resources

ISBN: 9781838645359

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

AI Crash Course

by Hadelin de Ponteves

7

Welcome to Q-Learning

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.