Nishanth Anand - The permanent and transient framework for continual reinforcement learning

The ability to learn continually from new experiences is a hallmark of natural intelligence. In the mammalian brain, this ability is often attributed to Complementary Learning Systems (CLS), where a slow-learning neocortex acquires structured, generalizable knowledge, and a fast-learning hippocampus rapidly encodes new experiences to aid adaptation. These two systems are believed to provide a robust balance between stability and plasticity---a trade-off that is central to continual learning. Inspired by these principles, this talk presents the Permanent and Transient (PT) framework, a conceptual approach for estimating predictive knowledge in Continual Reinforcement Learning. Specifically, we decompose the agent's predictive knowledge, such as value functions and successor features---into permanent and transient components. The permanent component captures stable, long-term structure in the predictions, while the transient component rapidly adapts estimates to the current situation. We develop novel representations and update rules to efficiently learn this decomposition. We provide theoretical guarantees for these algorithms and demonstrate that our framework outperforms competitive baselines across a variety of continual learning benchmarks. Finally, we close the loop between RL and neuroscience by showing that the PT framework offers a normative explanation for the dopamine ramping phenomenon. Nishanth Anand is a Ph.D. candidate in Computer Science at McGill University and Mila, supervised by Prof. Doina Precup. His research focuses on Continual Reinforcement Learning, where he develops novel algorithms inspired by cognitive science and neuroscience to help AI agents adapt to non-stationary environments. Grounded in both theoretical rigour and practical utility, Nishanth has authored first-author papers at NeurIPS and ICML. Beyond his research, he has co-instructed graduate-level RL courses at McGill and Polytechnique Montreal, served as a lead organizer for the