Eugene Vinitsky - Robust Autonomy Emerges from Self Play

Self-play has powered breakthroughs in two-player and multi-player games. Here we show that self-play is a surprisingly effective strategy in another domain. We show that robust and naturalistic driving emerges entirely from self-play in simulation at unprecedented scale -- 1.6~billion~km of driving. This is enabled by Gigaflow, a batched simulator that can synthesize and train on 42 years of subjective driving experience per hour on a single 8-GPU node. The resulting policy achieves state-of-the-art performance on three independent autonomous driving benchmarks. The policy outperforms the prior state of the art when tested on recorded real-world scenarios, amidst human drivers, without ever seeing human data during training. The policy is realistic when assessed against human references and achieves unprecedented robustness, averaging 17.5 years of continuous driving between incidents in simulation. Eugene is an Assistant Professor at NYU Tandon based in Civil Engineering with a PhD in control from UC Berkeley with Prof Alexandre Bayen. His research goal is to see complex, human-like behavior emerge from unsupervised interaction between groups of learning agents with an applications focus on robotics and transportation. Concretely, this leads to a lot of questions that he is currently interested in: How can we use RL to design models of human agents? How can we ensure that RL designed agents are human-compatible? How can we synthesize environments that push and test the capabilities of our agents? What algorithmic advances and software tools are needed to address these questions? In practice, this means working on understanding how to push the state of the art in multi-agent RL algorithms, designing new data-driven simulators, and trying to deploy simulator-designed controllers into real-world systems. He has worked at Apple, Tesla, DeepMind, and Facebook AI Research in the past. He is also a recipient of an NSF fellowship. This session is brought to you by th