Super Data Science ML & AI Podcast with Jon Krohn Insights (Audio)

Listen to Dr. Jon Krohn’s compelling insights into essential data science, artificial intelligence, and machine learning topics.

Series Description

In these AI Insights episodes, Dr. Jon Krohn deep dives solo into the most important topics in machine learning, AI, and other critical data science subfields.

Whether you’re a data science newbie, a seasoned professional, or simply fascinated by the power of AI, this podcast is for you. Jon explores cutting-edge techniques, and uncovers the practical applications of data analysis, predictive modeling, and machine learning. Get ready to transform your career and change the world with data!

Related Learning

Book: Deep Learning Illustrated: A Visual, Interactive Guide to Artificial Intelligence

Video: Deep Learning with TensorFlow, Keras, and PyTorch

Video: The Essential Machine Learning Foundations: Math, Probability, Statistics, and Computer Science (Video Collection)

Live Training by Jon Krohn

Playlist: Jon Krohn’s AI Catalyst Playlist

The AI Scientist: Towards Fully Automated, Open-Ended Scientific Discovery

Jon investigates published findings from the startup Sakana AI and its paper’s co-authors from the University of Oxford, the University of British Columbia and the Vector Institute in Toronto. These authors explore the potential of The AI Scientist, a framework that could change the way we conduct scientific research forever.

Mentioned in this episode:

• “The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery” by Chris Lu, Cong Lu, Robert Tjarko Lange, Jakob Foerster, Jeff Clune, David Ha
• OpenAI’s Japanese Rival Gets $1 Billion Valuation from Silicon Valley Investors
• “Research AI model unexpectedly modified its own code to extend runtime” by Benj Edwards
• SakanaAI/AI-Scientist on GitHub

Multi-Agent Systems: How Teams of LLMs Excel at Complex Tasks

Jon discusses the systems that are working to bridge the remaining gaps left by the latest large language models (LLMs).

Mentioned in this episode:

• Her
• GPT-4o
• Project Astra
• AutoGen
• Camel

Mixtral 8x22B: SOTA Open-Source LLM Capabilities at a Fraction of the Compute

Mixtral 8x22B takes the spotlight in this episode. Join Jon as he unveils how the French startup, Mistral, is pushing the boundaries of artificial intelligence with their latest innovation in Large Language Models.

Mentioned in this episode:

• Mistral
• Mixtral
• MMLU
• Llama2-70B
• Apache 2.0 license

Q*: OpenAI’s Rumored AGI Breakthrough

How long will it be until we reach artificial general intelligence, and will regulations over AI be effective in slowing its approach? In this episode, Jon peeks behind the curtains of OpenAI, where development of the world’s first model that can solve complex, nonlinear logical problems, Q*, might be well underway.

Mentioned in this episode:

• “OpenAI researchers warned board of AI breakthrough ahead of CEO ouster, sources say” Reuters
• “OpenAI Made an AI Breakthrough Before Altman Firing, Stoking Excitement and Concern” The Information
• “Training Verifiers to Solve Math Word Problems”, OpenAI
• Noam Brown’s tweet on OpenAI
• Yann LeCun’s tweet on Q*
• Playing Atari with Deep Reinforcement Learning
• Deep Q Learning Networks
• Jon Krohn Cartpole DQN
• AlphaGo
• Tree of Thoughts: Deliberate Problem Solving with Large Language Models
• NP-hard Problem
• The real research behind the wild rumors about OpenAI’s Q* project
• Jon Krohn's Hands-On Deep Reinforcement Learning Video Course
• Deep Learning Illustrated by Jon Krohn

Use Contrastive Search to get Human-Quality LLM Outputs

Learn how to achieve human-like outputs from LLMs. The new wave of generative AI can take on far more complex tasks via a number of decoding methods. The question is: Which of these methods best serves our needs? In this episode, Jon walks through the options currently available to us: GREEDY SEARCH, BEAM SEARCH, SAMPLING, and CONTRASTIVE SEARCH.

Mentioned in this episode:

• Two minutes NLP — Most used Decoding Methods for Language Models
• A Contrastive Framework for Neural Text Generation
• Contrastive Search GitHub repo
• Contrastive Search Is What You Need for Neural Text Generation
• Generating Human-level Text with Contrastive Search in Transformers
• Nebula

Six Reasons Why Building LLM Products Is Tricky

Jon breaks down Phillip Carter’s recent article on honeycomb.io, “All the Hard Stuff Nobody Talks About when Building Products with LLMs”. Jon highlights the risks underlying all six issues, from context windows through prompt engineering to compliance.

Mentioned in this episode:

• All the Hard Stuff Nobody Talks About when Building Products with LLMs
• Phillip Carter
• Nebula
• Prompt injection explained, with video, slides, and a transcript

SparseGPT: Remove 100 Billion Parameters but Retain 100% Accuracy

Have you heard of SparseGPT? This one-shot pruning technique downsizes large language models by 50% while maintaining 100% accuracy. Jon fills you in on everything you need to know.

Mentioned in this episode:

• IST Austria paper on SparseGPT
• Shaan Khosla
• Shaan Khosla’s Let’s Talk Text Substack newsletter

The Five Levels of AGI

How close are we to achieving artificial general intelligence, and how can we catalog our progress so far? This is the question that Google’s DeepMind recently sought to solve in their latest paper, “Levels of AGI: Operationalizing Progress on the Path to AGI.” Listen to the episode to hear Jon lay out the examples of both narrow and general AI for each level (where those levels have already been achieved!), and how this paper’s classifying structure helps us ascertain how close we are to reaching the levels of AGI we have not yet reached, from Level 2 onwards.

Mentioned in this episode:

• Levels of AGI: Operationalizing Progress on the Path to AGI
• Richards
• Bard
• Gemini
• Llama 2
• Siri
• Amazon Alexa
• IBM Watson
• AlphaGo
• AlphaFold

Subword Tokenization with Byte-Pair Encoding

Jon delivers a mini-bootcamp on tokenization, comparing word tokenization, character tokenization, and subword tokenization. 10-minutes is all you need to better understand this NLP-related process.

Mentioned in this episode:

• Word2Vec
• GloVe
• ELMo
• Byte-pair encoding
• BERT
• GPT-3
• XLNet
• Shaan Khosla
• Let’s Talk Text, Shaan Khosla’s Substack newsletter

OpenAI Whisper: General-Purpose Speech Recognition

Jon reviews OpenAI’s latest model, Whisper. This tool will vastly improve the way human speech is recognized and converted to text. Jon gets under the hood to show how the team managed to get such a powerfully accurate recognition model.

Mentioned in this episode:

• Whisper
• OpenAI Whisper GitHub
• Robust Speech Recognition via Large-Scale Weak Supervision
• wav2vec
• Whisper sampling
• Shaan Khosla
• Let’s Talk Text, Shaan Khosla’s Newsletter

The Five Levels of Self-Driving Cars

Self-driving cars are transforming the way we think about driving. This episode covers the five levels of automation, from basic assistance to full autonomy, and what they mean for the future of transportation.

Mentioned in this episode:

• Super Data Science Podcast AI Insights Episode: The Five Levels of AGI
• Taxonomy and Definitions for Terms Related to Driving Automation Systems for On-Road Motor Vehicles J3016_202104
• Waymo
• SAE International

AI x Solar Power = Abundant Energy

In this episode, Jon talks about the amazing rise of solar energy since AT&T Bell Labs introduced solar cells 70 years ago. What started as a battery replacement for remote areas has turned into a global phenomenon, with solar panels now covering an area the size of Jamaica and providing about 6% of the world's electricity. This rapid growth is impressive, with solar capacity doubling every three years. If this trend continues, by 2034, solar power could meet 60% of the world's electricity needs. Jon explains why this growth is happening so fast, and he explores how AI and data science can further advance solar energy and combat climate change.

Mentioned in this episode:

• The exponential growth of solar power will change the world

Deep Utopia: AI Could Solve All Human Problems in Our Lifetime

In this episode, Jon talks “books” as he outlines two nonfiction works on AI and futurism by Nick Bostrom. The Oxford philosopher’s latest book, Deep Utopia, is spotlighted in this episode. In it, Bostrom considers a future where artificial intelligence has solved humanity’s most profound problems. How will society be impacted? What are the risks of progressing beyond a point of no return when it comes to AI?

Mentioned in this episode:

• Superintelligence by Nick Bostrom
• Deep Utopia by Nick Bostrom

RFM-1 Gives Robots Human-like Reasoning and Conversation Abilities

RFM-1 takes the spotlight in this episode, where Jon delves into the intersection of AI and robotics. Spearheaded by Covariant and A.I. robotics luminary Pieter Abbeel, RFM-1 stands as a testament to the fusion of artificial intelligence advancements with robotic technologies, promising to redefine efficiency and autonomy in industrial applications.

Mentioned in this episode:

• Covariant
• Introducing RFM-1: Giving robots human-like reasoning capabilities
• NVIDIA Announces Project GR00T Foundation Model for Humanoid Robots and Major Isaac Robotics Platform Update
• NVIDIA GTC conference
• Figure Raises $675M for Its Humanoid Robot Development
• RFM-1's language capabilities
• RFM-1's physics capabilities
• Gemini Ultra
• Claude 3
• GPT-4

Vonnegut’s Player Piano (1952): An Eerie Novel on the Current AI Revolution

In this episode, host Jon Krohn explores the intriguing parallels between Kurt Vonnegut's debut novel, "Player Piano," and the contemporary advancements in artificial intelligence. As a long-time Vonnegut enthusiast, Jon reflects on the 1952 dystopian masterpiece's links to recent AI developments, and he explores how Vonnegut's narrative resonates with current discussions in machine learning and AI.

Mentioned in this episode:

• Kurt Vonnegut
• Player Piano by Kurt Vonnegut
• TD-Gammon
• Go
• Sora
• Diplomacy
• ChatGPT
• H2O.ai

The Mamba Architecture: Superior to Transformers in LLMs

Mamba, the innovative architecture poised to redefine the AI and deep learning arena, takes the spotlight this week. Jon explores the "Mamba: Linear-Time Sequence Modeling with Selective State Spaces" paper, a pioneering work by researchers at Carnegie Mellon and Princeton, shedding light on why Mamba is generating such excitement for its potential to reshape the AI landscape.

Mentioned in this episode:

• “Attention Is All You Need” by Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, Illia Polosukhin
• GPT-4
• Gemini Ultra
• "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding" by Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova
• DALL-E 3
• “Mamba: Linear-Time Sequence Modeling w/Selective State Spaces” by Albert Gu, Tri Dao
• Mamba Github repo
• Mamba-Chat

AlphaGeometry: AI is Suddenly as Capable as the Brightest Math Minds

In this episode, host Jon Krohn looks into developments from DeepMind, Google’s ground-breaking AI lab. Jon explains how the system has mastered deep mathematical reasoning. Listen to why this is a critical step towards making AI-generated solutions more accessible and understandable to humans and what it spells for the future of science.

Mentioned in this episode:

• “Solving olympiad geometry without human demonstrations” by Trieu H. Trinh, Yuhai Wu, Quoc V. Le, He He & Thang Luong
• AlphaGeometry
• FunSearch: Making new discoveries in mathematical sciences using Large Language Models
• Thinking, Fast and Slow by Daniel Kahneman
• “AlphaGeometry: An Olympiad-level AI system for geometry” by Trieu Trinh and Thang Luong

Jon Krohn is Co-Founder and Chief Data Scientist at the machine learning company Nebula. He authored the book Deep Learning Illustrated, an instant #1 bestseller that was translated into seven languages. He is also the host of SuperDataScience, the data science industry’s most listened-to podcast. Jon is renowned for his compelling lectures, which he offers at leading universities and conferences, as well as via his award-winning YouTube channel. He holds a PhD from Oxford and has been publishing on machine learning in prominent academic journals since 2010.

About Pearson Audio Learning

Pearson publishes expert-led audio learning so you can dip into today’s most important tech topics—wherever you are, wherever you go. Listen to industry leaders share first-hand experience, insights, and best practices using essential and emerging technologies.