Skip to Content
View all events

Machine Learning Foundations: Optimization

Published by Pearson

Beginner to intermediate content levelBeginner to intermediate

The [Machine Learning Foundations](https://learning.oreilly.com/search/?query=%22Machine%20Learning%20Foundations%22&formats=live online training) series of online trainings provides a comprehensive overview of all of the subjects -- mathematics, statistics, and computer science -- that underlie contemporary machine learning techniques, including deep learning and other artificial intelligence approaches.

All of the classes in the series bring theory to life through the combination of vivid full-color illustrations, straightforward Python examples within hands-on Jupyter notebook demos, and comprehension exercises with fully-worked solutions.

The focus is on providing you with a practical, functional understanding of the content covered. Context will be given for each topic, highlighting its relevance to machine learning. You will be better-positioned to understand cutting-edge machine learning papers and you will be provided with resources for digging even deeper into topics that pique your curiosity.

The eight classes in the series are organized into four couplets:

1. Linear Algebra

2. Calculus

3. Statistics

4. Computer Science

The content in the second class of each couplet follows directly from the content of the first, however you’re most welcome to pick and choose between any of the individual classes based on your particular interests or your existing familiarity with the material. (Note that at any given time, only a subset of these classes will be scheduled and open for registration. To be pushed notifications of upcoming classes in the series, sign up for the instructor’s email newsletter at jonkrohn.com.)

This class, Optimization, is the eighth of eight classes in the Machine Learning Foundations series. It builds upon the material from each of the other classes in the series -- on linear algebra, calculus, probability, statistics, and algorithms -- in order to provide a detailed introduction to training machine learning models. Through the measured exposition of theory paired with interactive examples, you’ll develop a working understanding of all of the essential theory behind the ubiquitous gradient descent approach to optimization as well as how to apply it yourself -- both at a granular, matrix operations level and a quick, abstract level -- with TensorFlow and PyTorch. You’ll also learn about the latest optimizers, such as Adam and Nadam, that are widely-used for training deep neural networks.

What you’ll learn and how you can apply it

  • Discover how the statistical and machine learning approaches to optimization differ, and why you would select one or the other for a given problem you’re solving.
  • Find out how the extremely versatile (stochastic) gradient descent optimization algorithm works, including how to apply it -- from a low, in-depth level as well as from a high, abstracted level -- within the most popular deep learning libraries, Tensorflow and PyTorch
  • Get acquainted with the “fancy” optimizers that are available for advanced machine learning approaches (e.g., deep learning) and when you should consider using them.

This live event is for you because...

  • You use high-level software libraries (e.g., scikit-learn, Keras, TensorFlow) to train or deploy machine learning algorithms, and would now like to understand the fundamentals underlying the abstractions, enabling you to expand your capabilities
  • You’re a software developer who would like to develop a firm foundation for the deployment of machine learning algorithms into production systems
  • You’re a data scientist who would like to reinforce your understanding of the subjects at the core of your professional discipline
  • You’re a data analyst or A.I. enthusiast who would like to become a data scientist or data/ML engineer, and so you’re keen to deeply understand the field you’re entering from the ground up (very wise of you!)

Prerequisites

  • All code demos will be in Python so experience with it or another object-oriented programming language would be helpful for following along with the hands-on examples.

Materials, downloads, or Supplemental Content needed in advance

  • During class, we’ll work on Jupyter notebooks interactively in the cloud via Google Colab. This requires zero setup and instructions will be provided in class.

Schedule

The time frames are only estimates and may vary according to how the class is progressing.

Segment 1: The Machine Learning Approach to Optimization (40 min)

  • The Statistical Approach to Regression: Ordinary Least Squares
  • When Statistical Approaches to Optimization Breakdown
  • The Machine Learning Solution
  • Q&A and Break

Segment 2: Gradient Descent (120 min)

  • Objective Functions
  • Cost / Loss / Error Functions
  • Minimizing Cost with Gradient Descent
  • The Global Minimum and Local Minima
  • Saddle Points
  • Maximizing Reward with Gradient Ascent
  • Learning Rate
  • Point-by-Point Regression in TensorFlow / PyTorch
  • Tensor-Based Regression in TensorFlow / PyTorch
  • Mini-Batches and Stochastic Gradient Descent (SGD)
  • Regression with SGD in TensorFlow / PyTorch
  • The Jacobian and the Hessian
  • Q&A and Break

Segment 3: Fancy Deep Learning Optimizers (50 min)

  • Momentum
  • Nesterov Momentum
  • AdaGrad
  • AdaDelta
  • RMSProp
  • Adam
  • Nadam
  • A Layer of Artificial Neurons in TensorFlow / PyTorch
  • Training a Deep Neural Net with SGD
  • Final Exercises and Q&A

Your Instructor

Jon Krohn

Jon Krohn is Co-Founder of the AI software firm Y Carrot and a Fellow at Lightning AI. He authored the book Deep Learning Illustrated, an instant #1 bestseller that was translated into seven languages. He is also the host of SuperDataScience, the data science industry’s most listened-to podcast. Jon is renowned for his compelling lectures, which he offers at leading universities and conferences, as well as via his award-winning YouTube channel. He holds a PhD from Oxford and has been publishing on machine learning in prominent academic journals since 2010.

linkedinXlinksearch

Skill covered

Machine Learning