George Hotz | Programming | wednesday at tiny corp. ASTs, imagenet, and triton | from $1499 comma 3

Name: George Hotz | Programming | wednesday at tiny corp. ASTs, imagenet, and triton | from $1499 comma 3
Uploaded: 2023-01-26T15:31:47Z
Channel: george hotz archive
Description: Date of stream 25 Jan 2023. from $1499 buy https://comma.ai/shop/comma-three Live-stream chat added as Subtitles/CC - English (Twitch Chat) - three-dot ...

george hotz archive · Beginner ·📰 AI News & Updates ·3y ago

Date of stream 25 Jan 2023. from $1499 buy https://comma.ai/shop/comma-three Live-stream chat added as Subtitles/CC - English (Twitch Chat) - three-dot menu icon - Show transcript Source files: - https://github.com/geohot/tinygrad - https://americanaffairsjournal.org/2021/08/the-value-of-nothing-capital-versus-growth/ Follow for notifications: - https://twitch.tv/georgehotz Support George: - https://twitch.tv/subs/georgehotz Programming playlist: - https://www.youtube.com/playlist?list=PLzFUMGbVxlQs5s-LNAyKgcq5SL28ZLLKC Chapters: 00:00:00 intro 00:01:52 plan for today 00:03:10 preparing whit…

Watch on YouTube ↗ (saves to browser)

Chapters (49)

intro

1:52 plan for today

3:10 preparing whiteboard

8:35 using triton

9:15 comma contracting tinycorp

11:30 qualcomm, tinycorp vs pytorch 2.0

12:15 ai chips, triton

12:52 jim keller, tenstorrent, shapeways

14:15 no ugly kernels, vs nvidia

15:05 scaling with gpu's, writing the right code

16:40 tinygrad kernel

18:35 tensor.py

21:00 onnx.py

21:20 tinygrad.org

22:40 easy to add backend

24:35 net.svg

27:57 alex

28:30 tinycorp needs to make money

29:35 tinygrad stable diffusion output

30:20 mlops.py

30:50 lazy.py

31:30 tinygrad vs pytorch 2.0

32:50 pytorch 2.0

33:10 250 vs 15 primitive operators

34:15 can tiny grad autdiff through exp(M)

35:50 frame evaluation api

36:34 pytorch inductor

37:00 zygote ad framework

37:45 writing a backend for pytorch

38:30 thinking what to do

39:50 torch inductor

40:35 not logging in to github

41:35 hope for tinygrad

45:10 trade offs

46:00 compile.py

47:35 mps support

50:25 NVIDIA_TF32_OVERRIDE

51:15 gpu vs cuda

51:54 3080 ti, reversing the ANE

52:13 library should be fast

54:40 ssh to ubuntu computer

55:08 poll slow kernel debug, python 3.12 upgrade, triton port

58:10 slow kernel debug

1:18:00 invalid command queue

1:29:00 copilot not good, thinking what to write

1:35:30 correct and faster

1:39:30 flip extra dimension indexing for speed

1:40:30 reduce error

1:49:15 expla

Playlist

Uploads from george hotz archive · george hotz archive · 0 of 60

← Previous Next →

George Hotz | Programming | wednesday at tiny corp. ASTs, imagenet, and triton | from $1499 comma 3

Chapters (49)

Playlist

Lesson complete!