The Startup Powering The Data Behind AGI

Weights & Biases · Beginner ·🛡️ AI Safety & Ethics ·6mo ago
In this episode of Gradient Dissent, Lukas Biewald talks with the CEO & founder of Surge AI, the billion-dollar company quietly powering the next generation of frontier LLMs. They discuss Surge's origin story, why traditional data labeling is broken, and how their research-focused approach is reshaping how models are trained. You’ll hear why inter-annotator agreement fails in high-complexity tasks like poetry and math, why synthetic data is often overrated, and how Surge builds rich RL environments to stress-test agentic reasoning. They also go deep on what kinds of data will be critical to f…
Watch on YouTube ↗ (saves to browser)

Chapters (17)

Intro: Who is Edwin Chen?
3:40 The problem with early data labeling systems
6:20 Search ranking, clickbait, and product principles
10:05 Why Surge focused on high-skill, high-quality labeling
13:50 From Craigslist workers to a billion-dollar business
16:40 Scaling without funding and avoiding Silicon Valley status games
21:15 Why most human data platforms lack real tech
25:05 Detecting cheaters, liars, and low-quality labelers
28:30 Why inter-annotator agreement is a flawed metric
32:15 What makes a great poem? Not checkboxes
36:40 Measuring subjective quality rigorously
40:00 What types of data are becoming more important
44:15 Scientific collaboration and frontier research data
47:00 Multimodal data, Argentinian coding, and hyper-specificity
50:10 What's wrong with LMSYS and benchmark hacking
53:20 Personalization and taste in model behavior
56:00 Synthetic data vs.

Playlist

Uploads from Weights & Biases · Weights & Biases · 0 of 60

← Previous Next →
1 0. What is machine learning?
0. What is machine learning?
Weights & Biases
 2 1. Build Your First Machine Learning Model
1. Build Your First Machine Learning Model
Weights & Biases
 3 Intro to ML: Course Overview
Intro to ML: Course Overview
Weights & Biases
 4 2. Multi-Layer Perceptrons
2. Multi-Layer Perceptrons
Weights & Biases
 5 3. Convolutional Neural Networks
3. Convolutional Neural Networks
Weights & Biases
 6 Weights & Biases at OpenAI
Weights & Biases at OpenAI
Weights & Biases
 7 Why Experiment Tracking is Crucial to OpenAI
Why Experiment Tracking is Crucial to OpenAI
Weights & Biases
 8 4. Autoencoders
4. Autoencoders
Weights & Biases
 9 5. Sentiment Analysis
5. Sentiment Analysis
Weights & Biases
 10 6. Recurrent Neural Networks [RNNs]
6. Recurrent Neural Networks [RNNs]
Weights & Biases
 11 7. Text Generation using LSTMs and GRUs
7. Text Generation using LSTMs and GRUs
Weights & Biases
 12 8. Text Classification Using Convolutional Neural Networks
8. Text Classification Using Convolutional Neural Networks
Weights & Biases
 13 9. Hybrid LSTMs [Long Short-Term Memory]
9. Hybrid LSTMs [Long Short-Term Memory]
Weights & Biases
 14 Toyota Research Institute on Experiment Tracking with Weights & Biases
Toyota Research Institute on Experiment Tracking with Weights & Biases
Weights & Biases
 15 Weights and Biases - Developer Tools for Deep Learning
Weights and Biases - Developer Tools for Deep Learning
Weights & Biases
 16 Introducing Weights & Biases
Introducing Weights & Biases
Weights & Biases
 17 10. Seq2Seq Models
10. Seq2Seq Models
Weights & Biases
 18 11. Transfer Learning for Domain-Specific Image Classification with Small Datasets
11. Transfer Learning for Domain-Specific Image Classification with Small Datasets
Weights & Biases
 19 12. One-shot learning for teaching neural networks to classify objects never seen before
12. One-shot learning for teaching neural networks to classify objects never seen before
Weights & Biases
 20 13. Speech Recognition with Convolutional Neural Networks in Keras/TensorFlow
13. Speech Recognition with Convolutional Neural Networks in Keras/TensorFlow
Weights & Biases
 21 14. Data Augmentation | Keras
14. Data Augmentation | Keras
Weights & Biases
 22 15. Batch Size and Learning Rate in CNNs
15. Batch Size and Learning Rate in CNNs
Weights & Biases
 23 Applied Deep Learning Fellowship Overview and Project Selection with Josh Tobin (2019)
Applied Deep Learning Fellowship Overview and Project Selection with Josh Tobin (2019)
Weights & Biases
 24 Grading Rubric for AI Applications with Sergey Karayev (2019)
Grading Rubric for AI Applications with Sergey Karayev (2019)
Weights & Biases
 25 16. Video Frame Prediction using CNNs and LSTMs (2019)
16. Video Frame Prediction using CNNs and LSTMs (2019)
Weights & Biases
 26 Image to LaTeX - Applied Deep Learning Fellowship (2019)
Image to LaTeX - Applied Deep Learning Fellowship (2019)
Weights & Biases
 27 17. Build and Deploy an Emotion Classifier (2019)
17. Build and Deploy an Emotion Classifier (2019)
Weights & Biases
 28 Applied Deep Learning - Data Management with Josh Tobin (2019)
Applied Deep Learning - Data Management with Josh Tobin (2019)
Weights & Biases
 29 Snorkel: Programming Training Data with Paroma Varma of Stanford University (2019)
Snorkel: Programming Training Data with Paroma Varma of Stanford University (2019)
Weights & Biases
 30 Applied Deep Learning - Troubleshooting and Debugging with Josh Tobin (2019)
Applied Deep Learning - Troubleshooting and Debugging with Josh Tobin (2019)
Weights & Biases
 31 Troubleshooting and Iterating ML Models with Lee Redden (2019)
Troubleshooting and Iterating ML Models with Lee Redden (2019)
Weights & Biases
 32 Designing a Machine Learning Project with Neal Khosla (2019)
Designing a Machine Learning Project with Neal Khosla (2019)
Weights & Biases
 33 Lukas Beiwald on ML Tools and Experiment Management (2019)
Lukas Beiwald on ML Tools and Experiment Management (2019)
Weights & Biases
 34 Building Machine Learning Teams with Josh Tobin (2019)
Building Machine Learning Teams with Josh Tobin (2019)
Weights & Biases
 35 Pieter Abeel on Potential Deep Learning Research Directions (2019)
Pieter Abeel on Potential Deep Learning Research Directions (2019)
Weights & Biases
 36 Testing and Deployment of Deep Learning Models with Josh Tobin (2019)
Testing and Deployment of Deep Learning Models with Josh Tobin (2019)
Weights & Biases
 37 Five Lessons for Team-Oriented Research with Peter Welder (2019)
Five Lessons for Team-Oriented Research with Peter Welder (2019)
Weights & Biases
 38 Applied Deep Learning - Rosanne Liu on AI Research (2019)
Applied Deep Learning - Rosanne Liu on AI Research (2019)
Weights & Biases
 39 Making the Mid-career Leap from Urban Design to Deep Learning/Data Science
Making the Mid-career Leap from Urban Design to Deep Learning/Data Science
Weights & Biases
 40 Organizing ML projects — W&B walkthrough (2020)
Organizing ML projects — W&B walkthrough (2020)
Weights & Biases
 41 Brandon Rohrer — Machine Learning in Production for Robots
Brandon Rohrer — Machine Learning in Production for Robots
Weights & Biases
 42 Nicolas Koumchatzky — Machine Learning in Production for Self-Driving Cars
Nicolas Koumchatzky — Machine Learning in Production for Self-Driving Cars
Weights & Biases
 43 My experiments with Reinforcement Learning with Jariullah Safi
My experiments with Reinforcement Learning with Jariullah Safi
Weights & Biases
 44 Applications of Machine Learning to COVID-19 Research with Isaac Godfried
Applications of Machine Learning to COVID-19 Research with Isaac Godfried
Weights & Biases
 45 VDLS Lavanya Product Walkthrough
VDLS Lavanya Product Walkthrough
Weights & Biases
 46 Testing Machine Learning Models with Eric Schles
Testing Machine Learning Models with Eric Schles
Weights & Biases
 47 How Linear Algebra is not like Algebra with Charles Frye
How Linear Algebra is not like Algebra with Charles Frye
Weights & Biases
 48 Predicting Protein Structures using Deep Learning with Jonathan King
Predicting Protein Structures using Deep Learning with Jonathan King
Weights & Biases
 49 Rachael Tatman — Conversational AI and Linguistics
Rachael Tatman — Conversational AI and Linguistics
Weights & Biases
 50 Reformer by Han Lee
Reformer by Han Lee
Weights & Biases
 51 Sequence Models with Pujaa Rajan
Sequence Models with Pujaa Rajan
Weights & Biases
 52 GitHub Actions & Machine Learning Workflows with Hamel Husain
GitHub Actions & Machine Learning Workflows with Hamel Husain
Weights & Biases
 53 Look Mom, No Indices! Vector Calculus with the Fréchet Derivative by Charles Frye
Look Mom, No Indices! Vector Calculus with the Fréchet Derivative by Charles Frye
Weights & Biases
 54 Jack Clark — Building Trustworthy AI Systems
Jack Clark — Building Trustworthy AI Systems
Weights & Biases
 55 Surprising Utility of Surprise: Why ML Uses Negative Log Probabilities - Charles Frye
Surprising Utility of Surprise: Why ML Uses Negative Log Probabilities - Charles Frye
Weights & Biases
 56 Track your machine learning experiments locally, with W&B Local - Chris Van Pelt
Track your machine learning experiments locally, with W&B Local - Chris Van Pelt
Weights & Biases
 57 Antipatterns in open source research code with Jariullah Safi
Antipatterns in open source research code with Jariullah Safi
Weights & Biases
 58 Attention for time series forecasting & COVID predictions - Isaac Godfried
Attention for time series forecasting & COVID predictions - Isaac Godfried
Weights & Biases
 59 Made with ML - Goku Mohandas
Made with ML - Goku Mohandas
Weights & Biases
 60 Angela & Danielle — Designing ML Models for Millions of Consumer Robots
Angela & Danielle — Designing ML Models for Millions of Consumer Robots
Weights & Biases
EXACTLY What You'll Learn Your 1st Year Blogging
Next Up
EXACTLY What You'll Learn Your 1st Year Blogging
Income School