Neural Network Programming with TensorFlow

Neural Network Programming with TensorFlow

by Manpreet Singh Ghotra, Rajdeep Dua
Released November 2017
Publisher(s): Packt Publishing
ISBN: 9781788390392

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

Neural Networks and their implementation decoded with TensorFlow

About This Book

  • Develop a strong background in neural network programming from scratch, using the popular Tensorflow library.
  • Use Tensorflow to implement different kinds of neural networks ? from simple feedforward neural networks to multilayered perceptrons, CNNs, RNNs and more.
  • A highly practical guide including real-world datasets and use-cases to simplify your understanding of neural networks and their implementation.

Who This Book Is For

This book is meant for developers with a statistical background who want to work with neural networks. Though we will be using TensorFlow as the underlying library for neural networks, book can be used as a generic resource to bridge the gap between the math and the implementation of deep learning. If you have some understanding of Tensorflow and Python and want to learn what happens at a level lower than the plain API syntax, this book is for you.

What You Will Learn

  • Learn Linear Algebra and mathematics behind neural network.
  • Dive deep into Neural networks from the basic to advanced concepts like CNN, RNN Deep Belief Networks, Deep Feedforward Networks.
  • Explore Optimization techniques for solving problems like Local minima, Global minima, Saddle points
  • Learn through real world examples like Sentiment Analysis.
  • Train different types of generative models and explore autoencoders.
  • Explore TensorFlow as an example of deep learning implementation.

In Detail

If you're aware of the buzz surrounding the terms such as "machine learning," "artificial intelligence," or "deep learning," you might know what neural networks are. Ever wondered how they help in solving complex computational problem efficiently, or how to train efficient neural networks? This book will teach you just that.

You will start by getting a quick overview of the popular TensorFlow library and how it is used to train different neural networks. You will get a thorough understanding of the fundamentals and basic math for neural networks and why TensorFlow is a popular choice Then, you will proceed to implement a simple feed forward neural network. Next you will master optimization techniques and algorithms for neural networks using TensorFlow. Further, you will learn to implement some more complex types of neural networks such as convolutional neural networks, recurrent neural networks, and Deep Belief Networks. In the course of the book, you will be working on real-world datasets to get a hands-on understanding of neural network programming. You will also get to train generative models and will learn the applications of autoencoders.

By the end of this book, you will have a fair understanding of how you can leverage the power of TensorFlow to train neural networks of varying complexities, without any hassle. While you are learning about various neural network implementations you will learn the underlying mathematics and linear algebra and how they map to the appropriate TensorFlow constructs.

Style and Approach

This book is designed to give you just the right number of concepts to back up the examples. With real-world use cases and problems solved, this book is a handy guide for you. Each concept is backed by a generic and real-world problem, followed by a variation, making you independent and able to solve any problem with neural networks. All of the content is demystified by a simple and straightforward approach.

Table of contents

  1. Title Page
  2. Copyright
    1. Neural Network Programming with TensorFlow
  3. Credits
  4. About the Authors
  5. About the Reviewer
  6. www.PacktPub.com
    1. Why subscribe?
  7. Customer Feedback
  8. Preface
    1. What this book covers
    2. What you need for this book
    3. Who this book is for
    4. Conventions
    5. Reader feedback
    6. Customer support
      1. Downloading the example code
      2. Downloading the color images of this book
      3. Errata
      4. Piracy
      5. Questions
  9. Maths for Neural Networks
    1. Understanding linear algebra
      1. Environment setup
        1. Setting up the Python environment in Pycharm
      2. Linear algebra structures
        1. Scalars, vectors, and matrices
        2. Tensors
      3. Operations
        1. Vectors
        2. Matrices
        3. Matrix multiplication
        4. Trace operator
        5. Matrix transpose
        6. Matrix diagonals
        7. Identity matrix
        8. Inverse matrix
      4. Solving linear equations
      5. Singular value decomposition
      6. Eigenvalue decomposition
      7. Principal Component Analysis
    2. Calculus
      1. Gradient
      2. Hessian
      3. Determinant
    3. Optimization
      1. Optimizers
    4. Summary
  10. Deep Feedforward Networks
    1. Defining feedforward networks
    2. Understanding backpropagation
    3. Implementing feedforward networks with TensorFlow
    4. Analyzing the Iris dataset
      1. Code execution
    5. Implementing feedforward networks with images
      1. Analyzing the effect of activation functions on the feedforward networks accuracy
    6. Summary
  11. Optimization for Neural Networks
    1. What is optimization?
    2. Types of optimizers
    3. Gradient descent
      1. Different variants of gradient descent
      2. Algorithms to optimize gradient descent
    4. Which optimizer to choose
      1. Optimization with an example
    5. Summary
  12. Convolutional Neural Networks
    1. An overview and the intuition of CNN
      1. Single Conv Layer Computation
      2. CNN in TensorFlow
        1. Image loading in TensorFlow
    2. Convolution operations
      1. Convolution on an image
      2. Strides
    3. Pooling
      1. Max pool
      2. Example code
        1. Average pool
    4. Image classification with convolutional networks
      1. Defining a tensor for input images and the first convolution layer
        1. Input tensor
        2. First convolution layer
        3. Second convolution layer
        4. Third convolution layer
        5. Flatten the layer
        6. Fully connected layers
        7. Defining cost and optimizer
        8. Optimizer
        9. First epoch
        10. Plotting filters and their effects on an image
    5. Summary
  13. Recurrent Neural Networks
    1. Introduction to RNNs
      1. RNN implementation
        1. Computational graph
      2. RNN implementation with TensorFlow
        1. Computational graph
    2. Introduction to long short term memory networks
      1. Life cycle of LSTM
      2. LSTM implementation
        1. Computational graph
    3. Sentiment analysis
      1. Word embeddings
      2. Sentiment analysis with an RNN
        1. Computational graph
    4. Summary
  14. Generative Models
    1. Generative models
      1. Discriminative versus generative models
      2. Types of generative models
        1. Autoencoders
        2. GAN
        3. Sequence models
    2. GANs
      1. GAN with an example
      2. Types of GANs
        1. Vanilla GAN
        2. Conditional GAN
        3. Info GAN
        4. Wasserstein GAN
        5. Coupled GAN
    3. Summary
  15. Deep Belief Networking
    1. Understanding deep belief networks
      1. DBN implementation
        1. Class initialization
        2. RBM class
          1. Pretraining the DBN
    2. Model training
    3. Predicting the label
    4. Finding the accuracy of the model
    5. DBN implementation for the MNIST dataset
      1. Loading the dataset
      2. Input parameters for a DBN with 256-Neuron RBM layers
      3. Output for a DBN with 256-neuron RBN layers
    6. Effect of the number of neurons in an RBM layer in a DBN
      1. An RBM layer with 512 neurons
      2. An RBM layer with 128 neurons
      3. Comparing the accuracy metrics
    7. DBNs with two RBM layers
    8. Classifying the NotMNIST dataset with a DBN
    9. Summary
  16. Autoencoders
    1. Autoencoder algorithms
    2. Under-complete autoencoders
    3. Dataset
    4. Basic autoencoders
      1. Autoencoder initialization
      2. AutoEncoder class
      3. Basic autoencoders with MNIST data
        1. Basic autoencoder plot of weights
        2. Basic autoencoder recreated images plot
      4. Basic autoencoder full code listing
      5. Basic autoencoder summary
    5. Additive Gaussian Noise autoencoder
      1. Autoencoder class
      2. Additive Gaussian Autoencoder with the MNIST dataset
        1. Training the model
        2. Plotting the weights
      3. Plotting the reconstructed images
      4. Additive Gaussian autoencoder full code listing
      5. Comparing basic encoder costs with the Additive Gaussian Noise autoencoder
      6. Additive Gaussian Noise autoencoder summary
    6. Sparse autoencoder
      1. KL divergence
        1. KL divergence in TensorFlow
          1. Cost of a sparse autoencoder based on KL Divergence
      2. Complete code listing of the sparse autoencoder
      3. Sparse autoencoder on MNIST data
      4. Comparing the Sparse encoder with the Additive Gaussian Noise encoder
    7. Summary
  17. Research in Neural Networks
    1. Avoiding overfitting in neural networks
      1. Problem statement
      2. Solution
      3. Results
    2. Large-scale video processing with neural networks
      1. Resolution improvements
      2. Feature histogram baselines
      3. Quantitative results
    3. Named entity recognition using a twisted neural network
      1. Example of a named entity recognition
      2. Defining Twinet
      3. Results
    4. Bidirectional RNNs
      1. BRNN on TIMIT dataset
    5. Summary
  18. Getting started with TensorFlow
    1. Environment setup
    2. TensorFlow comparison with Numpy
    3. Computational graph
      1. Graph
      2. Session objects
      3. Variables
      4. Scope
      5. Data input
      6. Placeholders and feed dictionaries
    4. Auto differentiation
    5. TensorBoard

Product information

  • Title: Neural Network Programming with TensorFlow
  • Author(s): Manpreet Singh Ghotra, Rajdeep Dua
  • Release date: November 2017
  • Publisher(s): Packt Publishing
  • ISBN: 9781788390392