book

Python Deep Learning

by Valentino Zocca, Gianmario Spacagna, Daniel Slater, Peter Roelants

April 2017

Intermediate to advanced

406 pages

10h 15m

English

Packt Publishing

Read now

Unlock full access

eBooks, discount offers, and moreWhy subscribe?
What this book covers

Downloading the example codeDownloading the color images of this book
PiracyQuestions
What is machine learning?
Supervised learningUnsupervised learningReinforcement learningSteps Involved in machine learning systemsBrief description of popular techniques/algorithmsLinear regressionDecision treesK-meansNaïve BayesSupport vector machinesThe cross-entropy methodNeural networksDeep learningApplications in real lifeA popular open source package
Why neural networks?
Neurons and layersDifferent types of activation functionThe back-propagation algorithmLinear regressionLogistic regressionBack-propagationApplications in industrySignal processingMedicalAutonomous car drivingBusinessPattern recognitionSpeech productionCode example of a neural network for the function xor
What is deep learning?Fundamental conceptsFeature learningDeep learning algorithms
Speech recognitionObject recognition and classification
TheanoTensorFlowKerasSample deep neural net code using Keras
AutoencodersNetwork designRegularization techniques for autoencodersDenoising autoencodersContractive autoencodersSparse autoencodersSummary of autoencoders
Hopfield networks and Boltzmann machinesBoltzmann machineRestricted Boltzmann machineImplementation in TensorFlowDeep belief networks
Similarities between artificial and biological models
Stride and padding in convolutional layers
Recurrent neural networksRNN — how to implement and trainBackpropagation through timeVanishing and exploding gradientsLong short term memory
Word-based modelsN-gramsNeural language modelsCharacter-based modelPreprocessing and reading dataLSTM networkTrainingSamplingExample training
Speech recognition pipelineSpeech as input dataPreprocessingAcoustic modelDeep belief networksRecurrent neural networksCTCAttention-based modelsDecodingEnd-to-end models
Early game playing AI
A supervised learning approach to games
Q-function
Experience replayEpsilon greedy
Atari Breakout random benchmarkPreprocessing the screenCreating a deep convolutional networkConvergence issues in Q-learningPolicy gradients versus Q-learning
Baseline for variance reductionGeneralized advantage estimator
What is anomaly and outlier detection?
Data modelingDetection modeling
Getting started with H2O
MNIST digit anomaly recognitionElectrocardiogram pulse detection
What is a data product?
Weights initializationParallel SGD using HOGWILD!Adaptive learningRate annealingMomentumNesterov's accelerationNewton's methodAdagradAdadeltaDistributed learning via Map/ReduceSparkling Water
Labeled DataUnlabeled DataSummary of validation
A/B TestingA summary of testing
POJO model exportAnomaly score APIsA summary of deployment

Content preview from Python Deep Learning

Chapter 6. Recurrent Neural Networks and Language Models

The neural network architectures we discussed in the previous chapters take in fixed sized input and provide fixed sized output. Even the convolutional networks used in image recognition (Chapter 5, Image Recognition) are flattened into a fixed output vector. This chapter will lift us from this constraint by introducing Recurrent Neural Networks (RNNs). RNNs help us deal with sequences of variable length by defining a recurrence relation over these sequences, hence the name.

The ability to process arbitrary sequences of input makes RNNs applicable for tasks such as language modeling (see section on Language Modelling) or speech recognition (see section on Speech Recognition). In fact, in ...