State Space Models w/ Albert Gu & Karan Goel (Cartesia AI)

The AI Epiphany ยท Beginner ยท๐Ÿ“ ML Fundamentals ยท1y ago
Become a Patreon: https://www.patreon.com/theaiepiphany ๐Ÿ‘จโ€๐Ÿ‘ฉโ€๐Ÿ‘งโ€๐Ÿ‘ฆ Join our Discord community: https://discord.gg/peBrCpheKE Albert & Karan joined us to talk state space models. They are the authors of the seminal S4 paper, Albert is the main author of thee Mamba paper - and together they're pioneering the SSM research and competing against transformer supremacy! :) Recently they posted super cool demos demonstrating just how much more real-time these models are compared to the baselines. โ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌ check out Cartesia AI: https://cartesia.ai/ โ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌ โŒš๏ธ Tiโ€ฆ
Watch on YouTube โ†— (saves to browser)

Chapters (11)

00:01:56 Intro
1:56 00:03:37 Hyperstack GPUs! (sponsored)
3:37 00:09:45 Cartesia AI, real-time intelligence
9:45 00:12:05 text-to-speech demo
12:05 00:36:35 state space models (SSMs)
36:35 00:38:30 bitter lessons transformers vs SSMs
38:30 00:41:30 free lunch? what are the actual tradeoffs?
41:30 00:42:55 Computing SSMs convolutionally
42:55 00:50:00 What is the state (RNNs, Attention, Conv, Neural ODEs)
50:00 00:55:30 Mamba 2, discrete vs continuous spectrum
55:30 01:03:39 Q&A: scaling laws

Playlist

Uploads from Aleksa Gordiฤ‡ - The AI Epiphany ยท Aleksa Gordiฤ‡ - The AI Epiphany ยท 0 of 60

โ† Previous Next โ†’
1 Intro | Neural Style Transfer #1
Intro | Neural Style Transfer #1
Aleksa Gordiฤ‡ - The AI Epiphany
 2 Basic Theory | Neural Style Transfer #2
Basic Theory | Neural Style Transfer #2
Aleksa Gordiฤ‡ - The AI Epiphany
 3 Optimization method | Neural Style Transfer #3
Optimization method | Neural Style Transfer #3
Aleksa Gordiฤ‡ - The AI Epiphany
 4 Advanced Theory | Neural Style Transfer #4
Advanced Theory | Neural Style Transfer #4
Aleksa Gordiฤ‡ - The AI Epiphany
 5 Anyone can make deepfakes now!
Anyone can make deepfakes now!
Aleksa Gordiฤ‡ - The AI Epiphany
 6 What is Computer Vision? | The Art of Creating Seeing Machines
What is Computer Vision? | The Art of Creating Seeing Machines
Aleksa Gordiฤ‡ - The AI Epiphany
 7 Feed-forward method | Neural Style Transfer #5
Feed-forward method | Neural Style Transfer #5
Aleksa Gordiฤ‡ - The AI Epiphany
 8 Alan Turing | Computing Machinery and Intelligence
Alan Turing | Computing Machinery and Intelligence
Aleksa Gordiฤ‡ - The AI Epiphany
 9 Feed-forward method (training) | Neural Style Transfer #6
Feed-forward method (training) | Neural Style Transfer #6
Aleksa Gordiฤ‡ - The AI Epiphany
 10 What is Google Deep Dream? (Basic Theory) | Deep Dream Series #1
What is Google Deep Dream? (Basic Theory) | Deep Dream Series #1
Aleksa Gordiฤ‡ - The AI Epiphany
 11 Semantic Segmentation in PyTorch | Neural Style Transfer #7
Semantic Segmentation in PyTorch | Neural Style Transfer #7
Aleksa Gordiฤ‡ - The AI Epiphany
 12 How to get started with Machine Learning
How to get started with Machine Learning
Aleksa Gordiฤ‡ - The AI Epiphany
 13 How to learn PyTorch? (3 easy steps) | 2021
How to learn PyTorch? (3 easy steps) | 2021
Aleksa Gordiฤ‡ - The AI Epiphany
 14 PyTorch or TensorFlow?
PyTorch or TensorFlow?
Aleksa Gordiฤ‡ - The AI Epiphany
 15 3 Machine Learning Projects For Beginners (Highly visual) | 2021
3 Machine Learning Projects For Beginners (Highly visual) | 2021
Aleksa Gordiฤ‡ - The AI Epiphany
 16 Machine Learning Projects (Intermediate level) | 2021
Machine Learning Projects (Intermediate level) | 2021
Aleksa Gordiฤ‡ - The AI Epiphany
 17 Cheapest (0$) Deep Learning Hardware Options | 2021
Cheapest (0$) Deep Learning Hardware Options | 2021
Aleksa Gordiฤ‡ - The AI Epiphany
 18 How to learn deep learning? (Transformers Example)
How to learn deep learning? (Transformers Example)
Aleksa Gordiฤ‡ - The AI Epiphany
 19 How do transformers work? (Attention is all you need)
How do transformers work? (Attention is all you need)
Aleksa Gordiฤ‡ - The AI Epiphany
 20 Developing a deep learning project (case study on transformer)
Developing a deep learning project (case study on transformer)
Aleksa Gordiฤ‡ - The AI Epiphany
 21 Vision Transformer (ViT) - An image is worth 16x16 words | Paper Explained
Vision Transformer (ViT) - An image is worth 16x16 words | Paper Explained
Aleksa Gordiฤ‡ - The AI Epiphany
 22 GPT-3 - Language Models are Few-Shot Learners | Paper Explained
GPT-3 - Language Models are Few-Shot Learners | Paper Explained
Aleksa Gordiฤ‡ - The AI Epiphany
 23 Google DeepMind's AlphaFold 2 explained! (Protein folding, AlphaFold 1, a glimpse into AlphaFold 2)
Google DeepMind's AlphaFold 2 explained! (Protein folding, AlphaFold 1, a glimpse into AlphaFold 2)
Aleksa Gordiฤ‡ - The AI Epiphany
 24 Attention Is All You Need (Transformer) | Paper Explained
Attention Is All You Need (Transformer) | Paper Explained
Aleksa Gordiฤ‡ - The AI Epiphany
 25 Graph Attention Networks (GAT) | GNN Paper Explained
Graph Attention Networks (GAT) | GNN Paper Explained
Aleksa Gordiฤ‡ - The AI Epiphany
 26 Graph Convolutional Networks (GCN) | GNN Paper Explained
Graph Convolutional Networks (GCN) | GNN Paper Explained
Aleksa Gordiฤ‡ - The AI Epiphany
 27 Graph SAGE - Inductive Representation Learning on Large Graphs | GNN Paper Explained
Graph SAGE - Inductive Representation Learning on Large Graphs | GNN Paper Explained
Aleksa Gordiฤ‡ - The AI Epiphany
 28 PinSage - Graph Convolutional Neural Networks for Web-Scale Recommender Systems | Paper Explained
PinSage - Graph Convolutional Neural Networks for Web-Scale Recommender Systems | Paper Explained
Aleksa Gordiฤ‡ - The AI Epiphany
 29 OpenAI CLIP - Connecting Text and Images | Paper Explained
OpenAI CLIP - Connecting Text and Images | Paper Explained
Aleksa Gordiฤ‡ - The AI Epiphany
 30 Temporal Graph Networks (TGN) | GNN Paper Explained
Temporal Graph Networks (TGN) | GNN Paper Explained
Aleksa Gordiฤ‡ - The AI Epiphany
 31 Graph Neural Network Project Update! (I'm coding GAT from scratch)
Graph Neural Network Project Update! (I'm coding GAT from scratch)
Aleksa Gordiฤ‡ - The AI Epiphany
 32 Graph Attention Network Project Walkthrough
Graph Attention Network Project Walkthrough
Aleksa Gordiฤ‡ - The AI Epiphany
 33 How to get started with Graph ML? (Blog walkthrough)
How to get started with Graph ML? (Blog walkthrough)
Aleksa Gordiฤ‡ - The AI Epiphany
 34 DQN - Playing Atari with Deep Reinforcement Learning | RL Paper Explained
DQN - Playing Atari with Deep Reinforcement Learning | RL Paper Explained
Aleksa Gordiฤ‡ - The AI Epiphany
 35 AlphaGo - Mastering the game of Go with deep neural networks and tree search | RL Paper Explained
AlphaGo - Mastering the game of Go with deep neural networks and tree search | RL Paper Explained
Aleksa Gordiฤ‡ - The AI Epiphany
 36 DeepMind's AlphaGo Zero and AlphaZero | RL paper explained
DeepMind's AlphaGo Zero and AlphaZero | RL paper explained
Aleksa Gordiฤ‡ - The AI Epiphany
 37 OpenAI - Solving Rubik's Cube with a Robot Hand | RL paper explained
OpenAI - Solving Rubik's Cube with a Robot Hand | RL paper explained
Aleksa Gordiฤ‡ - The AI Epiphany
 38 MuZero - Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model | RL Paper explained
MuZero - Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model | RL Paper explained
Aleksa Gordiฤ‡ - The AI Epiphany
 39 EfficientNetV2 - Smaller Models and Faster Training | Paper explained
EfficientNetV2 - Smaller Models and Faster Training | Paper explained
Aleksa Gordiฤ‡ - The AI Epiphany
 40 Implementing DeepMind's DQN from scratch! | Project Update
Implementing DeepMind's DQN from scratch! | Project Update
Aleksa Gordiฤ‡ - The AI Epiphany
 41 MLP-Mixer: An all-MLP Architecture for Vision | Paper explained
MLP-Mixer: An all-MLP Architecture for Vision | Paper explained
Aleksa Gordiฤ‡ - The AI Epiphany
 42 DeepMind's Android RL Environment - AndroidEnv
DeepMind's Android RL Environment - AndroidEnv
Aleksa Gordiฤ‡ - The AI Epiphany
 43 When Vision Transformers Outperform ResNets without Pretraining | Paper Explained
When Vision Transformers Outperform ResNets without Pretraining | Paper Explained
Aleksa Gordiฤ‡ - The AI Epiphany
 44 Non-Parametric Transformers | Paper explained
Non-Parametric Transformers | Paper explained
Aleksa Gordiฤ‡ - The AI Epiphany
 45 Chip Placement with Deep Reinforcement Learning | Paper Explained
Chip Placement with Deep Reinforcement Learning | Paper Explained
Aleksa Gordiฤ‡ - The AI Epiphany
 46 Text Style Brush - Transfer of text aesthetics from a single example | Paper Explained
Text Style Brush - Transfer of text aesthetics from a single example | Paper Explained
Aleksa Gordiฤ‡ - The AI Epiphany
 47 Graphormer - Do Transformers Really Perform Bad for Graph Representation? | Paper Explained
Graphormer - Do Transformers Really Perform Bad for Graph Representation? | Paper Explained
Aleksa Gordiฤ‡ - The AI Epiphany
 48 GANs N' Roses: Stable, Controllable, Diverse Image to Image Translation | Paper Explained
GANs N' Roses: Stable, Controllable, Diverse Image to Image Translation | Paper Explained
Aleksa Gordiฤ‡ - The AI Epiphany
 49 VQ-VAEs: Neural Discrete Representation Learning | Paper + PyTorch Code Explained
VQ-VAEs: Neural Discrete Representation Learning | Paper + PyTorch Code Explained
Aleksa Gordiฤ‡ - The AI Epiphany
 50 VQ-GAN: Taming Transformers for High-Resolution Image Synthesis | Paper Explained
VQ-GAN: Taming Transformers for High-Resolution Image Synthesis | Paper Explained
Aleksa Gordiฤ‡ - The AI Epiphany
 51 Multimodal Few-Shot Learning with Frozen Language Models | Paper Explained
Multimodal Few-Shot Learning with Frozen Language Models | Paper Explained
Aleksa Gordiฤ‡ - The AI Epiphany
 52 Focal Transformer: Focal Self-attention for Local-Global Interactions in Vision Transformers
Focal Transformer: Focal Self-attention for Local-Global Interactions in Vision Transformers
Aleksa Gordiฤ‡ - The AI Epiphany
 53 AudioCLIP: Extending CLIP to Image, Text and Audio | Paper Explained
AudioCLIP: Extending CLIP to Image, Text and Audio | Paper Explained
Aleksa Gordiฤ‡ - The AI Epiphany
 54 RMA: Rapid Motor Adaptation for Legged Robots | Paper Explained
RMA: Rapid Motor Adaptation for Legged Robots | Paper Explained
Aleksa Gordiฤ‡ - The AI Epiphany
 55 DALL-E: Zero-Shot Text-to-Image Generation | Paper Explained
DALL-E: Zero-Shot Text-to-Image Generation | Paper Explained
Aleksa Gordiฤ‡ - The AI Epiphany
 56 Channel Update: vacation, leaving Microsoft, approaching 10k subs and more!
Channel Update: vacation, leaving Microsoft, approaching 10k subs and more!
Aleksa Gordiฤ‡ - The AI Epiphany
 57 DETR: End-to-End Object Detection with Transformers | Paper Explained
DETR: End-to-End Object Detection with Transformers | Paper Explained
Aleksa Gordiฤ‡ - The AI Epiphany
 58 DINO: Emerging Properties in Self-Supervised Vision Transformers | Paper Explained!
DINO: Emerging Properties in Self-Supervised Vision Transformers | Paper Explained!
Aleksa Gordiฤ‡ - The AI Epiphany
 59 DeepMind DetCon: Efficient Visual Pretraining with Contrastive Detection | Paper Explained
DeepMind DetCon: Efficient Visual Pretraining with Contrastive Detection | Paper Explained
Aleksa Gordiฤ‡ - The AI Epiphany
 60 Do Vision Transformers See Like Convolutional Neural Networks? | Paper Explained
Do Vision Transformers See Like Convolutional Neural Networks? | Paper Explained
Aleksa Gordiฤ‡ - The AI Epiphany
80% of Engineering Isn't Coding
Next Up
80% of Engineering Isn't Coding
No Priors Podcast