book

Python Deep Learning

by Valentino Zocca, Gianmario Spacagna, Daniel Slater, Peter Roelants

April 2017

Intermediate to advanced

406 pages

10h 15m

English

Packt Publishing

Read now

Unlock full access

eBooks, discount offers, and moreWhy subscribe?
What this book covers

Downloading the example codeDownloading the color images of this book
PiracyQuestions
What is machine learning?
Supervised learningUnsupervised learningReinforcement learningSteps Involved in machine learning systemsBrief description of popular techniques/algorithmsLinear regressionDecision treesK-meansNaïve BayesSupport vector machinesThe cross-entropy methodNeural networksDeep learningApplications in real lifeA popular open source package
Why neural networks?
Neurons and layersDifferent types of activation functionThe back-propagation algorithmLinear regressionLogistic regressionBack-propagationApplications in industrySignal processingMedicalAutonomous car drivingBusinessPattern recognitionSpeech productionCode example of a neural network for the function xor
What is deep learning?Fundamental conceptsFeature learningDeep learning algorithms
Speech recognitionObject recognition and classification
TheanoTensorFlowKerasSample deep neural net code using Keras
AutoencodersNetwork designRegularization techniques for autoencodersDenoising autoencodersContractive autoencodersSparse autoencodersSummary of autoencoders
Hopfield networks and Boltzmann machinesBoltzmann machineRestricted Boltzmann machineImplementation in TensorFlowDeep belief networks
Similarities between artificial and biological models
Stride and padding in convolutional layers
Recurrent neural networksRNN — how to implement and trainBackpropagation through timeVanishing and exploding gradientsLong short term memory
Word-based modelsN-gramsNeural language modelsCharacter-based modelPreprocessing and reading dataLSTM networkTrainingSamplingExample training
Speech recognition pipelineSpeech as input dataPreprocessingAcoustic modelDeep belief networksRecurrent neural networksCTCAttention-based modelsDecodingEnd-to-end models
Early game playing AI
A supervised learning approach to games
Q-function
Experience replayEpsilon greedy
Atari Breakout random benchmarkPreprocessing the screenCreating a deep convolutional networkConvergence issues in Q-learningPolicy gradients versus Q-learning
Baseline for variance reductionGeneralized advantage estimator
What is anomaly and outlier detection?
Data modelingDetection modeling
Getting started with H2O
MNIST digit anomaly recognitionElectrocardiogram pulse detection
What is a data product?
Weights initializationParallel SGD using HOGWILD!Adaptive learningRate annealingMomentumNesterov's accelerationNewton's methodAdagradAdadeltaDistributed learning via Map/ReduceSparkling Water
Labeled DataUnlabeled DataSummary of validation
A/B TestingA summary of testing
POJO model exportAnomaly score APIsA summary of deployment

Content preview from Python Deep Learning

Speech recognition

In the previous sections, we saw how RNNs can be used to learn patterns of many different time sequences. In this section, we will look at how these models can be used for the problem of recognizing and understanding speech. We will give a brief overview of the speech recognition pipeline and provide a high-level view of how we can use neural networks in each part of the pipeline. In order to know more about the methods discussed in this section, we would like you to refer to the references.

Speech recognition pipeline

Speech recognition tries to find a transcription of the most probable word sequence considering the acoustic observations provided; this is represented by the following:

transcription = argmax( P(words | audio features)) ...