How to Keep Improving When You're Better Than Any Teacher - Iterated Distillation and Amplification

[2nd upload] AI systems can be trained using demonstrations from experts, but how do you train them to out-perform those experts? Can this still be done even without clear win/loss criteria? And how do you do it safely? This video was based on work including: "Supervising strong learners by amplifying weak experts" by Paul Christiano, Buck Shlegeris, Dario Amodei (https://arxiv.org/abs/1810.08575) https://openai.com/blog/amplifying-ai-training/ https://www.alignmentforum.org/s/EmDuGeRw749sD3GKd https://ai-alignment.com/iterated-distillation-and-amplification-157debfd1616 With thanks to my wonderful Patrons: ( https://www.patreon.com/robertskmiles ) Steef Jason Strack Jordan Medina Jason Hise Scott Worley JJ Hepboin Pedro A Ortega Said Polat Chris Canal Nicholas Kees Dupuis James Richárd Nagyfi Phil Moyer Alec Johnson Clemens Arbesser Bryce Daifuku Simon Strandgaard Jonatan R Michael Greve The Guru Of Vision Volodymyr David Tjäder Julius Brash Tom O'Connor Erik de Bruijn Robin Green Laura Olds Jon Halliday Paul Hobbs Jeroen De Dauw Tim Neilson Eric Scammell Igor Keller Ben Glanton Robert Sokolowski anul kumar sinha Jérôme Frossard Sean Gibat Sun Sun andrew Russell Cooper Lawton Gladamas Sylvain Chevalier DGJono robertvanduursen Dmitri Afanasjev Brian Sandberg Einar Ueland Marcel Ward Andrew Weir Taylor Smith Ben Archer Scott McCarthy Kabs Kabs Kabs Tendayi Mawushe Jannik Olbrich Anne Kohlbrenner Bjorn Nyblad Jussi Männistö Mr Fantastic Wr4thon Archy de Berker Marc Pauly Joshua Pratt Shevis Johnson Andy Kobre Brian Gillespie Martin Wind Peggy Youell Poker Chen Kees Darko Sperac Truls Paul Moffat Jelle Langen Anders Öhrt Marco Tiraboschi Michael Kuhinica Fraser Cain Robin Scharf Oren Milman John Rees Shawn Hartsock Seth Brothwell Brian Goodrich Clark Mitchell Kasper Schnack Michael Hunter Klemen Slavic Patrick Henderson Long Nguyen Oct todo22 Melisa Kostrzewski Hendrik Daniel Munter Graham Henry Duncan Orr