GPUs change everything

No Priors: AI, Machine Learning, Tech, & Startups · Advanced ·🏗️ Systems Design & Architecture ·7mo ago

Key Takeaways

Jensen Huang discusses how changes in computer architecture, such as the use of GPUs, require a complete rethink of algorithms, and how his company has adapted to design algorithms for various domains, including deep learning and neural networks, using tools like KDNN, a domain-specific language for accelerated deep learning.

Full Transcript

solving a problem relates to the computer architecture that's underneath. And if the architecture is CPUs connected through MPI and Ethernet or whatever it is, that algorithm is surely very different than thousands of processors connected by a fabric inside one GPU and thousands of GPUs inside a data [music] center. So obviously the algorithm has to be reframed and refactored. Our company got very good at designing computer algorithms. It could be for particle physics or fluid dynamics. And then of course one day it was related to deep learning and neural networks. And KDNN is essentially a domainspecific language for accelerated deep learning.

Original Description

Jensen Huang explains how changing computer architecture forces a complete rethink of algorithms. CPUs connected over ethernet demand one approach while thousands of GPU processors on a single fabric require another. #ai #tech #business #podcast
Watch on YouTube ↗ (saves to browser)
Sign in to unlock AI tutor explanation · ⚡30

Playlist

Uploads from No Priors: AI, Machine Learning, Tech, & Startups · No Priors: AI, Machine Learning, Tech, & Startups · 0 of 60

← Previous Next →
1 No Priors Ep. 13 | With Jensen Huang, Founder & CEO of NVIDIA
No Priors Ep. 13 | With Jensen Huang, Founder & CEO of NVIDIA
No Priors: AI, Machine Learning, Tech, & Startups
 2 No Priors Ep. 8 | With Neeva’s Sridhar Ramaswamy
No Priors Ep. 8 | With Neeva’s Sridhar Ramaswamy
No Priors: AI, Machine Learning, Tech, & Startups
 3 No Priors Ep. 7 | With Stanford Professor Dr. Percy Liang
No Priors Ep. 7 | With Stanford Professor Dr. Percy Liang
No Priors: AI, Machine Learning, Tech, & Startups
 4 No Priors Ep. 1 | With Noam Brown, Research Scientist at Meta
No Priors Ep. 1 | With Noam Brown, Research Scientist at Meta
No Priors: AI, Machine Learning, Tech, & Startups
 5 No Priors Ep. 9 | With Perplexity AI’s Aravind Srinivas and Denis Yarats
No Priors Ep. 9 | With Perplexity AI’s Aravind Srinivas and Denis Yarats
No Priors: AI, Machine Learning, Tech, & Startups
 6 No Priors Ep. 10 | With Copilot's Chief Architect and founder of Minion.AI Alex Graveley
No Priors Ep. 10 | With Copilot's Chief Architect and founder of Minion.AI Alex Graveley
No Priors: AI, Machine Learning, Tech, & Startups
 7 No Priors Ep. 11 | With Matei Zaharia, CTO of Databricks
No Priors Ep. 11 | With Matei Zaharia, CTO of Databricks
No Priors: AI, Machine Learning, Tech, & Startups
 8 No Priors Ep. 12 | With Noam Shazeer
No Priors Ep. 12 | With Noam Shazeer
No Priors: AI, Machine Learning, Tech, & Startups
 9 No Priors Ep. 14 | With Sarah Guo and Elad Gil
No Priors Ep. 14 | With Sarah Guo and Elad Gil
No Priors: AI, Machine Learning, Tech, & Startups
 10 No Priors Ep. 2 | With Runway ML’s Cristobal Valenzuela
No Priors Ep. 2 | With Runway ML’s Cristobal Valenzuela
No Priors: AI, Machine Learning, Tech, & Startups
 11 No Priors Ep. 3 | With Stability AI’s Emad Mostaque
No Priors Ep. 3 | With Stability AI’s Emad Mostaque
No Priors: AI, Machine Learning, Tech, & Startups
 12 No Priors Ep. 15 | With Kelvin Guu, Staff Research Scientist, Google Brain
No Priors Ep. 15 | With Kelvin Guu, Staff Research Scientist, Google Brain
No Priors: AI, Machine Learning, Tech, & Startups
 13 No Priors Ep. 4 | With Zipline’s Keller Rinaudo Cliffton
No Priors Ep. 4 | With Zipline’s Keller Rinaudo Cliffton
No Priors: AI, Machine Learning, Tech, & Startups
 14 No Priors Ep. 16 | With Mustafa Suleyman, Founder of DeepMind and Inflection
No Priors Ep. 16 | With Mustafa Suleyman, Founder of DeepMind and Inflection
No Priors: AI, Machine Learning, Tech, & Startups
 15 No Priors Ep. 17 | With Karan Singhal
No Priors Ep. 17 | With Karan Singhal
No Priors: AI, Machine Learning, Tech, & Startups
 16 No Priors Ep. 5 | With Huggingface’s Clem Delangue
No Priors Ep. 5 | With Huggingface’s Clem Delangue
No Priors: AI, Machine Learning, Tech, & Startups
 17 No Priors Ep. 6 | With Daphne Koller from Insitro
No Priors Ep. 6 | With Daphne Koller from Insitro
No Priors: AI, Machine Learning, Tech, & Startups
 18 No Priors Ep. 18 | With Kevin Scott, CTO of Microsoft
No Priors Ep. 18 | With Kevin Scott, CTO of Microsoft
No Priors: AI, Machine Learning, Tech, & Startups
 19 No Priors Ep. 19 | With Anduril CEO Brian Schimpf
No Priors Ep. 19 | With Anduril CEO Brian Schimpf
No Priors: AI, Machine Learning, Tech, & Startups
 20 No Priors Ep. 20 | With Sarah Guo and Elad Gil
No Priors Ep. 20 | With Sarah Guo and Elad Gil
No Priors: AI, Machine Learning, Tech, & Startups
 21 No Priors Ep. 21 | With Datadog Co-founder/CEO Olivier Pomel
No Priors Ep. 21 | With Datadog Co-founder/CEO Olivier Pomel
No Priors: AI, Machine Learning, Tech, & Startups
 22 No Priors Ep. 22 | With Instacart CEO Fidji Simo
No Priors Ep. 22 | With Instacart CEO Fidji Simo
No Priors: AI, Machine Learning, Tech, & Startups
 23 No Priors Ep. 23 | With Snowflake's CEO Frank Slootman
No Priors Ep. 23 | With Snowflake's CEO Frank Slootman
No Priors: AI, Machine Learning, Tech, & Startups
 24 No Priors Ep. 24 | With Devi Parikh from Meta
No Priors Ep. 24 | With Devi Parikh from Meta
No Priors: AI, Machine Learning, Tech, & Startups
 25 No Priors Ep. 25 | With Palantir's CTO Shyam Sankar
No Priors Ep. 25 | With Palantir's CTO Shyam Sankar
No Priors: AI, Machine Learning, Tech, & Startups
 26 No Priors Ep. 26 | With Weights & Biases CEO Lukas Biewald
No Priors Ep. 26 | With Weights & Biases CEO Lukas Biewald
No Priors: AI, Machine Learning, Tech, & Startups
 27 No Priors Ep. 27 | With Sarah Guo & Elad Gil
No Priors Ep. 27 | With Sarah Guo & Elad Gil
No Priors: AI, Machine Learning, Tech, & Startups
 28 No Priors Ep. 28 | With Khan Academy’s Creator Sal Khan
No Priors Ep. 28 | With Khan Academy’s Creator Sal Khan
No Priors: AI, Machine Learning, Tech, & Startups
 29 No Priors Ep. 28 | With Khan Academy’s Creator Sal Khan (Japanese Version)
No Priors Ep. 28 | With Khan Academy’s Creator Sal Khan (Japanese Version)
No Priors: AI, Machine Learning, Tech, & Startups
 30 No Priors Ep. 29 | With Inceptive CEO Jakob Uszkoreit
No Priors Ep. 29 | With Inceptive CEO Jakob Uszkoreit
No Priors: AI, Machine Learning, Tech, & Startups
 31 No Priors Ep. 30 | With Vercel CEO Guillermo Rauch
No Priors Ep. 30 | With Vercel CEO Guillermo Rauch
No Priors: AI, Machine Learning, Tech, & Startups
 32 No Priors Ep. 31 | With Cerebras CEO Andrew Feldman
No Priors Ep. 31 | With Cerebras CEO Andrew Feldman
No Priors: AI, Machine Learning, Tech, & Startups
 33 No Priors Ep. 32 | With NEAR’s Illia Polosukhin
No Priors Ep. 32 | With NEAR’s Illia Polosukhin
No Priors: AI, Machine Learning, Tech, & Startups
 34 No Priors Ep. 33 | With Replit's CEO & Co-Founder Amjad Masad
No Priors Ep. 33 | With Replit's CEO & Co-Founder Amjad Masad
No Priors: AI, Machine Learning, Tech, & Startups
 35 No Priors Ep. 34 | With Ginkgo Bioworks Co-Founder and CEO Jason Kelly
No Priors Ep. 34 | With Ginkgo Bioworks Co-Founder and CEO Jason Kelly
No Priors: AI, Machine Learning, Tech, & Startups
 36 No Priors Ep. 35 | With Sarah Guo and Elad Gil
No Priors Ep. 35 | With Sarah Guo and Elad Gil
No Priors: AI, Machine Learning, Tech, & Startups
 37 No Priors Ep. 36 | With Hubspot's Co-Founder Brian Halligan
No Priors Ep. 36 | With Hubspot's Co-Founder Brian Halligan
No Priors: AI, Machine Learning, Tech, & Startups
 38 No Priors Ep. 37 | With Kawal Gandhi
No Priors Ep. 37 | With Kawal Gandhi
No Priors: AI, Machine Learning, Tech, & Startups
 39 No Priors Ep. 38 | With Material Security Co-Founder Ryan Noon
No Priors Ep. 38 | With Material Security Co-Founder Ryan Noon
No Priors: AI, Machine Learning, Tech, & Startups
 40 No Priors Ep. 39 | With OpenAI Co-Founder & Chief Scientist Ilya Sutskever
No Priors Ep. 39 | With OpenAI Co-Founder & Chief Scientist Ilya Sutskever
No Priors: AI, Machine Learning, Tech, & Startups
 41 No Priors Ep. 40 | With Arthur Mensch, CEO Mistral AI
No Priors Ep. 40 | With Arthur Mensch, CEO Mistral AI
No Priors: AI, Machine Learning, Tech, & Startups
 42 No Priors Ep. 41 | With Imbue Co-Founders Kanjun Qiu and Josh Albrecht
No Priors Ep. 41 | With Imbue Co-Founders Kanjun Qiu and Josh Albrecht
No Priors: AI, Machine Learning, Tech, & Startups
 43 No Priors Ep. 42 | With Sarah Guo and Elad Gil
No Priors Ep. 42 | With Sarah Guo and Elad Gil
No Priors: AI, Machine Learning, Tech, & Startups
 44 No Priors Ep. 43 | With Clara Shih, CEO of Salesforce AI
No Priors Ep. 43 | With Clara Shih, CEO of Salesforce AI
No Priors: AI, Machine Learning, Tech, & Startups
 45 No Priors Ep. 44 | With Former Square CEO Alyssa Henry
No Priors Ep. 44 | With Former Square CEO Alyssa Henry
No Priors: AI, Machine Learning, Tech, & Startups
 46 No Priors Ep. 45 | With Reid Hoffman
No Priors Ep. 45 | With Reid Hoffman
No Priors: AI, Machine Learning, Tech, & Startups
 47 No Priors Ep. 46 | Best of 2023 with Sarah Guo and Elad Gil
No Priors Ep. 46 | Best of 2023 with Sarah Guo and Elad Gil
No Priors: AI, Machine Learning, Tech, & Startups
 48 No Priors Ep. 47 | With Sourcegraph CTO Beyang Liu
No Priors Ep. 47 | With Sourcegraph CTO Beyang Liu
No Priors: AI, Machine Learning, Tech, & Startups
 49 No Priors Ep. 48 | With Covariant CEO Peter Chen
No Priors Ep. 48 | With Covariant CEO Peter Chen
No Priors: AI, Machine Learning, Tech, & Startups
 50 No Priors Ep. 49 | With Shopify VP of Core Product Glen Coates
No Priors Ep. 49 | With Shopify VP of Core Product Glen Coates
No Priors: AI, Machine Learning, Tech, & Startups
 51 No Priors Ep. 50 | With Stripe Head of Information Emily Glassberg Sands
No Priors Ep. 50 | With Stripe Head of Information Emily Glassberg Sands
No Priors: AI, Machine Learning, Tech, & Startups
 52 No Priors Ep. 51 | With Notion CEO Ivan Zhao
No Priors Ep. 51 | With Notion CEO Ivan Zhao
No Priors: AI, Machine Learning, Tech, & Startups
 53 No Priors Ep. 52 | With Pinecone CEO Edo Liberty
No Priors Ep. 52 | With Pinecone CEO Edo Liberty
No Priors: AI, Machine Learning, Tech, & Startups
 54 No Priors Ep. 53 | With AMD CTO Mark Papermaster
No Priors Ep. 53 | With AMD CTO Mark Papermaster
No Priors: AI, Machine Learning, Tech, & Startups
 55 No Priors Ep. 54 | With Sarah Guo & Elad Gil
No Priors Ep. 54 | With Sarah Guo & Elad Gil
No Priors: AI, Machine Learning, Tech, & Startups
 56 No Priors Ep. 55 | With Figma CEO Dylan Field
No Priors Ep. 55 | With Figma CEO Dylan Field
No Priors: AI, Machine Learning, Tech, & Startups
 57 No Priors Ep 56 | With Baseten CEO and Co-Founder Tuhin Srivastava
No Priors Ep 56 | With Baseten CEO and Co-Founder Tuhin Srivastava
No Priors: AI, Machine Learning, Tech, & Startups
 58 No Priors Ep. 57 | With LangChain CEO and Co-Founder Harrison Chase
No Priors Ep. 57 | With LangChain CEO and Co-Founder Harrison Chase
No Priors: AI, Machine Learning, Tech, & Startups
 59 No Priors Ep. 58 | The argument for humanoid robots with Brett Adcock from Figure
No Priors Ep. 58 | The argument for humanoid robots with Brett Adcock from Figure
No Priors: AI, Machine Learning, Tech, & Startups
 60 No Priors Ep. 59 | With Sarah Guo & Elad Gil
No Priors Ep. 59 | With Sarah Guo & Elad Gil
No Priors: AI, Machine Learning, Tech, & Startups

This video discusses how changes in computer architecture, such as the use of GPUs, require a complete rethink of algorithms, and how to design and optimize algorithms for deep learning and neural networks. Jensen Huang shares his company's experience in adapting to new architectures and designing algorithms for various domains.

Key Takeaways
  1. Understand the impact of computer architecture on algorithm design
  2. Learn to design algorithms for GPUs and other specialized architectures
  3. Apply domain-specific languages like KDNN for accelerated deep learning
  4. Refactor algorithms for deep learning and neural networks
  5. Optimize algorithms for specific architectures
💡 The choice of computer architecture has a significant impact on the design of algorithms, and adapting to new architectures requires a complete rethink of algorithm design

Related Reads

Up next
9-Step Software Architect Roadmap 2026 | System Design | #shorts
SCALER
Watch →