George Hotz | Programming | can we fit a LLaMA inside a tinygrad? | $1499 comma.ai/shop/comma-three

Name: George Hotz | Programming | can we fit a LLaMA inside a tinygrad? | $1499 comma.ai/shop/comma-three
Uploaded: 2023-03-09T23:14:18Z
Channel: george hotz archive
Description: Date of stream 08 Mar 2023. from $1499 buy https://comma.ai/shop/comma-three Live-stream chat added as Subtitles/CC - English (Twitch Chat) - three-dot ...

george hotz archive · Beginner ·🧠 Large Language Models ·3y ago

Date of stream 08 Mar 2023. from $1499 buy https://comma.ai/shop/comma-three Live-stream chat added as Subtitles/CC - English (Twitch Chat) - three-dot menu icon - Show transcript Source files: - https://github.com/geohot/tinygrad/blob/llama/examples/llama.py - https://github.com/geohot/tinygrad/tree/llama - https://github.com/geohot/tinygrad Follow for notifications: - https://twitch.tv/georgehotz Support George: - https://twitch.tv/subs/georgehotz Programming playlist: - https://www.youtube.com/playlist?list=PLzFUMGbVxlQs5s-LNAyKgcq5SL28ZLLKC Chapters: 00:00:00 intro 00:01:30 guitar tuning…

Watch on YouTube ↗ (saves to browser)

Chapters (38)

intro

1:30 guitar tuning and playing

4:30 giving advice to elon, woke

7:25 llama, trip to india, bengaluru, bike to mumbai

8:40 when software was cool

11:20 llama, privilege

13:45 discord ban, pressure, power

16:05 prime, blue crown, money

18:00 tiny corp, making value, food

22:25 capitalism

23:50 facebook LLaMA, apply for access, torrent

25:10 ai safety, enjoy the decline

28:00 deepmind, google, rants

28:40 thinking for yourself, jesus, george is christian, god

30:30 a progression to christian

31:50 big text, copilot

35:10 running on gpu m1

39:30 using tinygrad tiny ram usage, float16

46:00 METAL=1

47:25 numpy size of datatype

49:12 interesting, np.frombuffer

52:10 problem raw buffers to have a datatype?

52:55 python read into buffer, not making copies = fast

59:40 loading from disk, making it lazy

1:00:50 tinygrad is the future, replacing programing with dsp paradigm

1:02:17 tinygrad has grown

1:03:50 metal buffer, float16 hard

1:05:20 interview question how big is a float

1:06:40 making metal buffer, supporting dtypes

1:09:40 tinygrad recognize llama

1:10:25 facebook llama arxiv, unbiased model

1:13:25 class Attention, FeedForward

1:21:40 washed heads

1:24:00 class RMSNorm

1:30:25 testing inference

1:35:30 error and slow loading because torch

1:39:00 wasting 0.6 seconds

1:40:45 how transformers work, Linear

Playlist

Uploads from george hotz archive · george hotz archive · 0 of 60

← Previous Next →

George Hotz | Programming | can we fit a LLaMA inside a tinygrad? | $1499 comma.ai/shop/comma-three

Chapters (38)

Playlist

Lesson complete!