Talk: Kernels Deep Dive (Ben Burtenshaw)

Hugging Face · Intermediate ·🧠 Large Language Models ·3w ago
In this talk, Ben Burtenshaw from Hugging Face breaks down why optimized kernels are critical for real-world deep learning performance and how the Hugging Face Kernels ecosystem makes them easier to build and use. He covers memory-bound bottlenecks, the kernel-builder workflow, reproducible multi-hardware builds with Nix, and practical PyTorch/Transformers integration patterns that reduce setup time from hours to seconds. ## Chapters 0:00 Intro and speaker background 1:35 Why Hugging Face Kernels matters 2:05 Compute vs memory bottlenecks in deep learning 3:30 Fused kernels and why the…
Watch on YouTube ↗ (saves to browser)

Chapters (19)

Intro and speaker background
1:35 Why Hugging Face Kernels matters
2:05 Compute vs memory bottlenecks in deep learning
3:30 Fused kernels and why they speed things up
5:05 Talk agenda and ecosystem overview
5:35 Kernel pain points: fragmentation and long installs
7:12 Supporting older, cheaper hardware for the community
8:18 Goal: from CMake errors to one-line kernel usage
8:54 Kernels + kernel-builder architecture
10:00 Reproducible builds with Nix and support matrix
11:45 Kernel project structure (`build.toml`, sources, torch extension)
12:23 Publishing kernels to the Hugging Face Hub
13:25 Real-world gain: faster FlashAttention setup
14:18 Docs, repos, and how to get started
16:00 Verifying compatibility and loading kernels in Python
17:20 Managing local cache with `hf cache ls`
17:55 Kernelizing PyTorch layers with hub mappings
19:32 Transformers integration (`use_kernels=True`)
20:48 Performance chart and closing resources

Playlist

Uploads from HuggingFace · HuggingFace · 0 of 60

← Previous Next →
1 The Future of Natural Language Processing
The Future of Natural Language Processing
HuggingFace
 2 Trends in Model Size & Computational Efficiency in NLP
Trends in Model Size & Computational Efficiency in NLP
HuggingFace
 3 Increasing Data Usage in Natural Language Processing
Increasing Data Usage in Natural Language Processing
HuggingFace
 4 In Domain & Out of Domain Generalization in the Future of NLP
In Domain & Out of Domain Generalization in the Future of NLP
HuggingFace
 5 The Limits of NLU & the Rise of NLG in the Future of NLP
The Limits of NLU & the Rise of NLG in the Future of NLP
HuggingFace
 6 The Lack of Robustness in the Future of NLP
The Lack of Robustness in the Future of NLP
HuggingFace
 7 Inductive Bias, Common Sense, Continual Learning in The Future of NLP
Inductive Bias, Common Sense, Continual Learning in The Future of NLP
HuggingFace
 8 Train and use a NLP model in 10 mins!
Train and use a NLP model in 10 mins!
HuggingFace
 9 Automatic text classification in a few lines of code
Automatic text classification in a few lines of code
HuggingFace
 10 Train a Hugging Face Transformers Model with Amazon SageMaker
Train a Hugging Face Transformers Model with Amazon SageMaker
HuggingFace
 11 What is Transfer Learning?
What is Transfer Learning?
HuggingFace
 12 The pipeline function
The pipeline function
HuggingFace
 13 Navigating the Model Hub
Navigating the Model Hub
HuggingFace
 14 Character-based tokenizers
Character-based tokenizers
HuggingFace
 15 Transformer models: Decoders
Transformer models: Decoders
HuggingFace
 16 The Transformer architecture
The Transformer architecture
HuggingFace
 17 Transformer models: Encoder-Decoders
Transformer models: Encoder-Decoders
HuggingFace
 18 Transformer models: Encoders
Transformer models: Encoders
HuggingFace
 19 Keras introduction
Keras introduction
HuggingFace
 20 The push to hub API
The push to hub API
HuggingFace
 21 Subword-based tokenizers
Subword-based tokenizers
HuggingFace
 22 Fine-tuning with TensorFlow
Fine-tuning with TensorFlow
HuggingFace
 23 Learning rate scheduling with TensorFlow
Learning rate scheduling with TensorFlow
HuggingFace
 24 TensorFlow Predictions and metrics
TensorFlow Predictions and metrics
HuggingFace
 25 Tokenizers Overview
Tokenizers Overview
HuggingFace
 26 Word-based tokenizers
Word-based tokenizers
HuggingFace
 27 Welcome to the Hugging Face course
Welcome to the Hugging Face course
HuggingFace
 28 The tokenization pipeline
The tokenization pipeline
HuggingFace
 29 Supercharge your PyTorch training loop with Accelerate
Supercharge your PyTorch training loop with Accelerate
HuggingFace
 30 The Trainer API
The Trainer API
HuggingFace
 31 Batching inputs together (PyTorch)
Batching inputs together (PyTorch)
HuggingFace
 32 Batching inputs together (TensorFlow)
Batching inputs together (TensorFlow)
HuggingFace
 33 Hugging Face Datasets overview (Pytorch)
Hugging Face Datasets overview (Pytorch)
HuggingFace
 34 Hugging Face Datasets overview (Tensorflow)
Hugging Face Datasets overview (Tensorflow)
HuggingFace
 35 What is dynamic padding?
What is dynamic padding?
HuggingFace
 36 What happens inside the pipeline function? (PyTorch)
What happens inside the pipeline function? (PyTorch)
HuggingFace
 37 What happens inside the pipeline function? (TensorFlow)
What happens inside the pipeline function? (TensorFlow)
HuggingFace
 38 Instantiate a Transformers model (PyTorch)
Instantiate a Transformers model (PyTorch)
HuggingFace
 39 Instantiate a Transformers model (TensorFlow)
Instantiate a Transformers model (TensorFlow)
HuggingFace
 40 Preprocessing sentence pairs (PyTorch)
Preprocessing sentence pairs (PyTorch)
HuggingFace
 41 Preprocessing sentence pairs (TensorFlow)
Preprocessing sentence pairs (TensorFlow)
HuggingFace
 42 Write your training loop in PyTorch
Write your training loop in PyTorch
HuggingFace
 43 Managing a repo on the Model Hub
Managing a repo on the Model Hub
HuggingFace
 44 Chapter 1 Live Session with Sylvain
Chapter 1 Live Session with Sylvain
HuggingFace
 45 Chapter 2 Live Session with Lewis
Chapter 2 Live Session with Lewis
HuggingFace
 46 The push to hub API
The push to hub API
HuggingFace
 47 Chapter 2 Live Session with Sylvain
Chapter 2 Live Session with Sylvain
HuggingFace
 48 Chapter 3 live sessions with Lewis (PyTorch)
Chapter 3 live sessions with Lewis (PyTorch)
HuggingFace
 49 Day 1 Talks: JAX, Flax & Transformers 🤗
Day 1 Talks: JAX, Flax & Transformers 🤗
HuggingFace
 50 Day 2 Talks: JAX, Flax & Transformers 🤗
Day 2 Talks: JAX, Flax & Transformers 🤗
HuggingFace
 51 Day 3 Talks JAX, Flax, Transformers 🤗
Day 3 Talks JAX, Flax, Transformers 🤗
HuggingFace
 52 Chapter 4 live sessions with Omar
Chapter 4 live sessions with Omar
HuggingFace
 53 Deploy a Hugging Face Transformers Model from S3 to Amazon SageMaker
Deploy a Hugging Face Transformers Model from S3 to Amazon SageMaker
HuggingFace
 54 Deploy a Hugging Face Transformers Model from the Model Hub to Amazon SageMaker
Deploy a Hugging Face Transformers Model from the Model Hub to Amazon SageMaker
HuggingFace
 55 Run a Batch Transform Job using Hugging Face Transformers and Amazon SageMaker
Run a Batch Transform Job using Hugging Face Transformers and Amazon SageMaker
HuggingFace
 56 [Webinar] How to add machine learning capabilities with just a few lines of code
[Webinar] How to add machine learning capabilities with just a few lines of code
HuggingFace
 57 Hugging Face + Zapier Demo Video
Hugging Face + Zapier Demo Video
HuggingFace
 58 Hugging Face + Google Sheets Demo
Hugging Face + Google Sheets Demo
HuggingFace
 59 Hugging Face Infinity Launch - 09/28
Hugging Face Infinity Launch - 09/28
HuggingFace
 60 Introducing AutoNLP (Trailer)
Introducing AutoNLP (Trailer)
HuggingFace
5 Levels of AI Agents - From Simple LLM Calls to Multi-Agent Systems
Next Up
5 Levels of AI Agents - From Simple LLM Calls to Multi-Agent Systems
Dave Ebbelaar (LLM Eng)