Hands-On Meta Learning with Python

Hands-On Meta Learning with Python

by Sudharsan Ravichandiran
Released December 2018
Publisher(s): Packt Publishing
ISBN: 9781789534207

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

Explore a diverse set of meta-learning algorithms and techniques to enable human-like cognition for your machine learning models using various Python frameworks

Key Features

  • Understand the foundations of meta learning algorithms
  • Explore practical examples to explore various one-shot learning algorithms with its applications in TensorFlow
  • Master state of the art meta learning algorithms like MAML, reptile, meta SGD

Book Description

Meta learning is an exciting research trend in machine learning, which enables a model to understand the learning process. Unlike other ML paradigms, with meta learning you can learn from small datasets faster.

Hands-On Meta Learning with Python starts by explaining the fundamentals of meta learning and helps you understand the concept of learning to learn. You will delve into various one-shot learning algorithms, like siamese, prototypical, relation and memory-augmented networks by implementing them in TensorFlow and Keras. As you make your way through the book, you will dive into state-of-the-art meta learning algorithms such as MAML, Reptile, and CAML. You will then explore how to learn quickly with Meta-SGD and discover how you can perform unsupervised learning using meta learning with CACTUs. In the concluding chapters, you will work through recent trends in meta learning such as adversarial meta learning, task agnostic meta learning, and meta imitation learning.

By the end of this book, you will be familiar with state-of-the-art meta learning algorithms and able to enable human-like cognition for your machine learning models.

What you will learn

  • Understand the basics of meta learning methods, algorithms, and types
  • Build voice and face recognition models using a siamese network
  • Learn the prototypical network along with its variants
  • Build relation networks and matching networks from scratch
  • Implement MAML and Reptile algorithms from scratch in Python
  • Work through imitation learning and adversarial meta learning
  • Explore task agnostic meta learning and deep meta learning

Who this book is for

Hands-On Meta Learning with Python is for machine learning enthusiasts, AI researchers, and data scientists who want to explore meta learning as an advanced approach for training machine learning models. Working knowledge of machine learning concepts and Python programming is necessary.

Publisher resources

Download Example Code

Table of contents

  1. Title Page
  2. Copyright and Credits
    1. Hands-On Meta Learning with Python
  3. Dedication
  4. About Packt
    1. Why subscribe?
    2. Packt.com
  5. Contributors
    1. About the author
    2. About the reviewers
    3. Packt is searching for authors like you
  6. Preface
    1. Who this book is for
    2. What this book covers
    3. To get the most out of this book
      1. Download the example code files
      2. Conventions used
    4. Get in touch
      1. Reviews
  7. Introduction to Meta Learning
    1. Meta learning
      1. Meta learning and few-shot
    2. Types of meta learning
      1. Learning the metric space
      2. Learning the initializations
      3. Learning the optimizer
    3. Learning to learn gradient descent by gradient descent
    4. Optimization as a model for few-shot learning
    5. Summary
    6. Questions
    7. Further reading
  8. Face and Audio Recognition Using Siamese Networks
    1. What are siamese networks?
      1. Architecture of siamese networks
      2. Applications of siamese networks
    2. Face recognition using siamese networks
    3. Building an audio recognition model using siamese networks
    4. Summary
    5. Questions
    6. Further readings
  9. Prototypical Networks and Their Variants
    1. Prototypical networks
      1. Algorithm
      2. Performing classification using prototypical networks
    2. Gaussian prototypical network
      1. Algorithm
    3. Semi-prototypical networks
    4. Summary
    5. Questions
    6. Further reading
  10. Relation and Matching Networks Using TensorFlow
    1. Relation networks
      1. Relation networks in one-shot learning
      2. Relation networks in few-shot learning
      3. Relation networks in zero-shot learning
      4. Loss function
    2. Building relation networks using TensorFlow
    3. Matching networks
      1. Embedding functions
        1. The support set embedding function (g)
        2. The query set embedding function (f)
    4. The architecture of matching networks
    5. Matching networks in TensorFlow
    6. Summary
    7. Questions
    8. Further reading
  11. Memory-Augmented Neural Networks
    1. NTM
      1. Reading and writing in NTM
        1. Read operation
        2. Write operation
          1. Erase operation
          2. Add operation
      2. Addressing mechanisms
        1. Content-based addressing
        2. Location-based addressing
          1. Interpolation
          2. Convolution shift
          3. Sharpening
    2. Copy tasks using NTM
    3. Memory-augmented neural networks (MANN)
      1. Read and write operations
        1. Read operation
        2. Write operation
    4. Summary
    5. Questions
    6. Further reading
  12. MAML and Its Variants
    1. MAML
      1. MAML algorithm
      2. MAML in supervised learning
        1. Building MAML from scratch
          1. Generate data points
          2. Single layer neural network
          3. Training using MAML
      3. MAML in reinforcement learning
    2. Adversarial meta learning
      1. FGSM
      2. ADML
      3. Building ADML from scratch
        1. Generating data points
        2. FGSM
        3. Single layer neural network
        4. Adversarial meta learning
    3. CAML
      1. CAML algorithm
    4. Summary
    5. Questions
    6. Further reading
  13. Meta-SGD and Reptile
    1. Meta-SGD
      1. Meta-SGD for supervised learning
        1. Building Meta-SGD from scratch
          1. Generating data points
          2. Single layer neural network
          3. Meta-SGD
      2. Meta-SGD for reinforcement learning
    2. Reptile
      1. The Reptile algorithm
      2. Sine wave regression using Reptile
        1. Generating data points
        2. Two-layered neural network
        3. Reptile
    3. Summary
    4. Questions
    5. Further readings
  14. Gradient Agreement as an Optimization Objective
    1. Gradient agreement as an optimization
      1. Weight calculation
      2. Algorithm
    2. Building gradient agreement algorithm with MAML
      1. Generating data points
      2. Single layer neural network
      3. Gradient agreement in MAML
    3. Summary
    4. Questions
    5. Further reading
  15. Recent Advancements and Next Steps
    1. Task agnostic meta learning (TAML)
      1. Entropy maximization/reduction
        1. Algorithm
      2. Inequality minimization
        1. Inequality measures
          1. Gini coefficient
          2. Theil index
          3. Variance of algorithms
        2. Algorithm
    2. Meta imitation learning
      1. MIL algorithm
    3. CACTUs
      1. Task generation using CACTUs
    4. Learning to learn in concept space
      1. Key components
        1. Concept generator
        2. Concept discriminator
        3. Meta learner
      2. Loss function
        1. Concept discrimination loss
        2. Meta learning loss
      3. Algorithm
    5. Summary
    6. Questions
    7. Further reading
  16. Assessments
    1. Chapter 1: Introduction to Meta Learning
    2. Chapter 2: Face and Audio Recognition Using Siamese Networks
    3. Chapter 3: Prototypical Networks and Their Variants
    4. Chapter 4: Relation and Matching Networks Using TensorFlow
    5. Chapter 5: Memory-Augmented Neural Networks
    6. Chapter 6: MAML and Its Variants
    7. Chapter 7: Meta-SGD and Reptile Algorithms
    8. Chapter 8: Gradient Agreement as an Optimization Objective
    9. Chapter 9: Recent Advancements and Next Steps
  17. Other Books You May Enjoy
    1. Leave a review - let other readers know what you think

Product information

  • Title: Hands-On Meta Learning with Python
  • Author(s): Sudharsan Ravichandiran
  • Release date: December 2018
  • Publisher(s): Packt Publishing
  • ISBN: 9781789534207