Planning with LLMs for Code Generation | ICLR 2023

75 Hard Research · Beginner ·📄 Research Papers Explained ·2y ago

About this lesson

#AIResearch #75HardResearch #75HardAI #ResearchPaperExplained The video discusses the ICLR 2023 paper "" that uses Monte Carlo Tree Search (MCTS), a planning algorithm, to do better LLM Decoding at inference time to generate competitive level code. Link to the other related videos: 1. Introduction to Reinforcement Learning and Planning (with running example) https://youtu.be/cLA-tojed0s 2. Multi-arm Bandits and Upper Confidence Bound (UCB) https://youtu.be/rm2HoK6KAbk 3. Monte Carlo Tree Search (MCTS) explained with a detailed example https://youtu.be/reoP5usaYU4 Chapters: 00:00 - Intro 00:38 - Code Generation Task 02:26 - Why standard LLM decoding can be bad for Code generation? 04:14 - What is Beam Search? 07:32 - Problem with Beam Search (with Example) 11:14 - MCTS for code generation 11:23 - Selection (MCTS step 1) 11:45 - Expansion (MCTS step 2) 13:03 - Evaluation (MCTS step 3) 15:38 - Backpropagation (MCTS step 4) 16:25 - How MCTS solve Beam search's problem? 18:04 - Final Algorithm 19:30 - Optimizations 21:30 - MCTS Node Selection (P-UCB)

Original Description

#AIResearch #75HardResearch #75HardAI #ResearchPaperExplained The video discusses the ICLR 2023 paper "" that uses Monte Carlo Tree Search (MCTS), a planning algorithm, to do better LLM Decoding at inference time to generate competitive level code. Link to the other related videos: 1. Introduction to Reinforcement Learning and Planning (with running example) https://youtu.be/cLA-tojed0s 2. Multi-arm Bandits and Upper Confidence Bound (UCB) https://youtu.be/rm2HoK6KAbk 3. Monte Carlo Tree Search (MCTS) explained with a detailed example https://youtu.be/reoP5usaYU4 Chapters: 00:00 - Intro 00:38 - Code Generation Task 02:26 - Why standard LLM decoding can be bad for Code generation? 04:14 - What is Beam Search? 07:32 - Problem with Beam Search (with Example) 11:14 - MCTS for code generation 11:23 - Selection (MCTS step 1) 11:45 - Expansion (MCTS step 2) 13:03 - Evaluation (MCTS step 3) 15:38 - Backpropagation (MCTS step 4) 16:25 - How MCTS solve Beam search's problem? 18:04 - Final Algorithm 19:30 - Optimizations 21:30 - MCTS Node Selection (P-UCB)

Watch on YouTube ↗ (saves to browser)

Sign in to unlock AI tutor explanation · ⚡30

Related Reads

A lightweight workflow for keeping up with AI conference papers

Learn a lightweight workflow to stay updated with AI conference papers and never miss important research again

Dev.to · Daniel

Why CitedEvidence Believes Great Researchers Read Less Than You Think

Great researchers don't read every paper, but rather focus on reading the right ones and applying their knowledge effectively

How to Write a Literature Review That Actually Argues Something

Learn to write a literature review that presents a clear argument, a crucial skill for ML researchers and students

Medium · Machine Learning

I Built a Personal Paper Engine to Stop Losing Research Papers

Build a personal paper engine to organize and annotate research papers efficiently

Dev.to · Ethan

Chapters (14)

Intro

0:38 Code Generation Task

2:26 Why standard LLM decoding can be bad for Code generation?

4:14 What is Beam Search?

7:32 Problem with Beam Search (with Example)

11:14 MCTS for code generation

11:23 Selection (MCTS step 1)

11:45 Expansion (MCTS step 2)

13:03 Evaluation (MCTS step 3)

15:38 Backpropagation (MCTS step 4)

16:25 How MCTS solve Beam search's problem?

18:04 Final Algorithm

19:30 Optimizations

21:30 MCTS Node Selection (P-UCB)

Thunderbit Review: AI Web Scraping in Just 2 Clicks 🔥