O'Reilly logo

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Apache Spark Deep Learning Recipes

Video Description

Over 35 videos that streamline deep learning in a distributed environment with Apache Spark

About This Video

  • Discover practical recipes for distributed deep learning with Apache Spark
  • Learn to use libraries such as Keras and TensorFlow
  • Predict Apple stock market cost with the LSTM model

In Detail

With deep learning gaining rapid mainstream adoption in modern-day industries, organizations are looking for ways to unite popular big data tools with highly efficient deep learning libraries. As a result, this will help deep learning models train with higher efficiency and speed.

This video course start offs by explaining the process of developing a neural network from scratch using deep learning libraries such as Tensorflow or Keras. It focuses on the pain points of convolution neural networks. We’ll predict fire department calls with Spark ML and Apple stock market cost with LSTM. We’ll walk you through the steps to classify chatbot conversation data for escalation.

By the end of the video course, you'll have all the basic knowledge about apache spark.

The code bundle for this video course is available at https://github.com/PacktPublishing/Apache-Spark-Deep-Learning-Recipes

Table of Contents

  1. Chapter 1 : Creating a Neural Network in Spark
    1. The Course overview 00:03:03
    2. Creating a Dataframes in Pyspark 00:03:36
    3. Manipulating Columns in a Pyspark Dataframes 00:01:34
    4. Converting a PySparkdataframe to an array 00:03:08
    5. Visualizing an Array in a Scatterplot 00:02:47
    6. Setting up Weights and Biases for Input into the Neural Network 00:03:20
    7. Normalizing the Input Data for the Neural Network 00:02:07
    8. Validating Array for Optimal Neural Network Performance 00:01:28
    9. Setting up the Activation Function with Sigmoid 00:02:44
    10. Creating the Sigmoid Derivative Function 00:01:41
    11. Calculating the Cost Function in a Neural Network 00:03:25
    12. Predicting Gender based on Height and Weight 00:01:57
    13. Visualizing Prediction Scores 00:01:26
  2. Chapter 2 : Pain Points of Convolutional Neural Networks
    1. Pain Point #1: Importing MNIST Images 00:05:01
    2. Pain Point #2: Visualizing MNIST Images 00:02:42
    3. Pain Point #3: Exporting MNIST Images as Files 00:01:21
    4. Pain Point #4: Augmenting MNIST Images 00:03:27
    5. Pain Point #5: Utilizing Alternate Sources for Trained Images 00:02:36
    6. Pain Point #6: Prioritizing High-Level Libraries for CNNs 00:07:07
  3. Chapter 3 : Predicting Fire Department Calls with Spark ML
    1. Downloading the San Francisco Fire Department Calls Dataset 00:03:56
    2. Identifying the Target Variable of the Logistic Regression Model 00:03:49
    3. Preparing Feature Variables for the Logistic Regression Model 00:03:49
    4. Applying the Logistic Regression Model 00:03:44
    5. Evaluating the Accuracy of the Logistic Regression Model 00:02:43
  4. Chapter 4 : Natural Language Processing with TF-IDF
    1. Downloading and Analyzing the Therapy Bot Session Dataset 00:04:13
    2. Visualizing Word Counts in the Dataset 00:01:36
    3. Calculating Sentiment Analysis of Text 00:03:22
    4. Removing Stop Words from the Text 00:02:52
    5. Training and Evaluating TF-IDF Model Performance 00:05:30
    6. Comparing Model Performance to a Baseline Score 00:01:36
  5. Chapter 5 : Predicting Apple Stock Market Cost with LSTM
    1. Downloading Stock Market Data for Apple 00:03:36
    2. Exploring and Visualizing Stock Market Data for Apple 00:04:26
    3. Preparing Stock Data for Model Performance 00:05:25
    4. Building the LSTM Model 00:02:29
    5. Evaluating the Model 00:01:58