GameGAN Explained!

This video explains the new Neural Game Engine GameGAN from researchers at NVIDIA! This paper uses Deep Learning to store Pacman inside of a learned world model such that you can play the game by sending actions to the generative neural network. This video will describe the problem and how the proposed solution through careful architecture and loss function design! Thanks for Watching! Please Subscribe! Paper Links: NVIDIA GameGAN Blog Post: https://blogs.nvidia.com/blog/2020/05/22/gamegan-research-pacman-anniversary/ NVIDIA Quick video presenting GameGAN: https://www.youtube.com/watch?v=BYt6r8z6pUY World Models: https://worldmodels.github.io/ GauGAN (SPADE layer) demo video: https://www.youtube.com/watch?v=p5U4NgVGAwg Four Novel Approaches to Manipulating Fabric: https://bair.berkeley.edu/blog/2020/05/05/fabrics/https://bair.berkeley.edu/blog/2020/05/05/fabrics/ Intuitively Understanding Variational Autoencoders: https://towardsdatascience.com/intuitively-understanding-variational-autoencoders-1bfe67eb5daf How much Knowledge Can You Pack into the Parameters of a Language Model? https://arxiv.org/pdf/2002.08910.pdf MuZero: https://deepmind.com/research/publications/Mastering-Atari-Go-Chess-and-Shogi-by-Planning-with-a-Learned-Model Neural Turing Machines: https://arxiv.org/pdf/1410.5401.pdf GAN Compression: https://arxiv.org/pdf/2003.08936.pdf CycleGAN: https://arxiv.org/pdf/1703.10593.pdf Yann LeCun's 2020 ICLR Keynote (Importance of multi-modal predictions mentioned in video): https://iclr.cc/virtual_2020/speaker_7.html Regularizing Trajectory Optimization with Denoising Autoencoders: https://papers.nips.cc/paper/8552-regularizing-trajectory-optimization-with-denoising-autoencoders.pdf 1:48 Neural Game Engines 2:40 Application to Model-Based RL 3:47 Predicting the Future: GANs, VAEs, and RNNs 4:44 Multi-Modal Predictions for the Future 5:33 GameGAN Architecture 6:43 Dynamics Engine 8:20 External Memory Module 11:47 Rendering Engine 14:20 Losses to train GameGAN