George Hotz | Programming | a tiny tour through tinygrad (noob lesson) | github.com/geohot/tinygrad

Date of stream 3 Jul 2022. Live-stream chat added as Subtitles/CC - English (Twitch Chat). Stream title: a tiny tour through tinygrad (noob lesson) Source files: - https://github.com/geohot/tinygrad Follow for notifications: - https://twitch.tv/georgehotz Support George: - https://twitch.tv/subs/georgehotz Programming playlist: - https://www.youtube.com/playlist?list=PLzFUMGbVxlQs5s-LNAyKgcq5SL28ZLLKC Compute computer - Ubuntu 20.04.4 LTS - AMD Ryzen 9 5950X - 64GB RAM - AMD Radeon RX 6900 XT Streaming computer - Apple MacBook M1 - LG UltraFine 5K - Blue Yeti - Apple Magic Keyboard - HHKB - tmux & Vim & Visual Studio Code with Vim Key Bindings and other https://github.com/geohot/configuration Chapters: 00:00:00 intro 00:00:41 drug test, drinks, to shower or not to 00:04:40 sub only chat, tinygrad lazy 00:05:10 tour of tinygrad, Tensor 00:06:20 graph 00:07:30 print underlying buffer if it's realized 00:09:20 lazy mode, two armies of haters 00:10:20 multiplying two matrices and graph 00:11:00 GPU support, dashes - not a contiguous tensor 00:11:35 tinygrad codebase, three backends CPU, GPU and Torch 00:11:50 lazy mode 00:13:00 tinygrad is a tensor library, tensor is a list of numbers 00:14:20 benefit of laziness 00:16:10 stream lag, dirty hoodie 00:16:48 ImageNet inference, stream lag, GPU=1 00:19:47 tinygrad definition, 992 lines 00:20:25 tensor class, karpathy, micrograd, 00:23:15 backends, openpilot model 00:24:05 automatic differentiation engine, bad bug 00:26:05 let's fix that bug 00:29:15 hlops, tests 00:31:50 fun thing in tinygrad 00:32:25 pad2d, matmul 00:35:40 transpose, flatten, sum 00:36:40 softmax, dropout, pools, conv, unary 00:37:05 binary ops 00:38:20 tinygrad is one of the best things I have ever written 00:39:25 div, short break 00:40:30 neural networks ops 00:41:45 linear 00:43:10 sequential, reduce 00:44:40 an instantiation of the function in the context 00:45:45 mlops, cleanup, unary ops 00:47:50 good tests, more cleanup 00:53:20 the derivative