George Hotz | Researching | RL is dumb and doesn't work (theory) | Reinforcement Learning | Part 3

Name: George Hotz | Researching | RL is dumb and doesn't work (theory) | Reinforcement Learning | Part 3
Uploaded: 2024-01-11T22:47:20Z
Channel: george hotz archive
Description: Date of the stream 8 Jan 2024. from $1250 buy https://comma.ai/shop/comma-3x & best ADAS system in the world https://openpilot.comma.ai Live-stream chat...

george hotz archive · Beginner ·🧠 Large Language Models ·2y ago

Date of the stream 8 Jan 2024. from $1250 buy https://comma.ai/shop/comma-3x & best ADAS system in the world https://openpilot.comma.ai Live-stream chat added as Subtitles/CC - English (Twitch Chat) - at the bottom - Show Transcript Sources: - https://openreview.net/pdf?id=9pe38WpsbX - http://www.incompleteideas.net/IncIdeas/BitterLesson.html tinygrad bounties: - https://docs.google.com/spreadsheets/d/1WKHbT-7KOgjEawq5h5Ic1qUWzpfAzuD_J06N1JwOCGs/ Follow for notifications: - https://twitch.tv/georgehotz Support George: - https://twitch.tv/subs/georgehotz Pre-order tinybox: - https://buy.stripe…

Watch on YouTube ↗ (saves to browser)

Chapters (40)

muted intro

0:20 un-muted

0:45 the incident

5:10 not recording stream locally

5:40 twitch vs twitter revenue

7:00 why is RL difficult, data

9:25 Richard S. Sutton rl book

10:20 every bad thing about data

11:35 deepmind soccer robot

12:55 neurolink, changing monkeys data

13:30 search always works

14:20 things to consider when debugging rl pytorch

14:50 beta in PPO

17:10 cleanup

20:50 episodes 40

25:10 beta

27:00 gamma discount factor

28:48 openpilot does not use any RL

30:55 committing changes to master

31:25 combined experience replay (CER)

33:50 go play with the code

34:35 debugging RL

35:05 don't try stuff and run to see if it works

35:40 ML going dark, companies do not share secrets

36:35 misguided belief, comma traning, compute

37:30 your ability to execute

38:10 5 year old reddit post

39:20 private data, curating the data

40:15 keeping tricks secret

40:45 upstream code open source, MIT licence

41:30 aixi, bellman equation

42:50 bayesian optimization search

43:40 ml empirical science

44:05 data extraction pipeline, training pipeline

44:55 human groundtruth

46:20 minimal rl test environments

48:00 batch size 256

49:00 spam trying programming

53:25 why it is so bad

54:50 openai spinning u

Playlist

Uploads from george hotz archive · george hotz archive · 0 of 60

← Previous Next →

George Hotz | Researching | RL is dumb and doesn't work (theory) | Reinforcement Learning | Part 3

Chapters (40)

Playlist

Lesson complete!