Mitigating Value Hallucination in Dyna Planning via Multistep Predecessor Models
📰 ArXiv cs.AI
Mitigating value hallucination in Dyna planning using multistep predecessor models improves sample efficiency in reinforcement learning
Action Steps
- Identify the potential causes of failure in Dyna agents
- Learn accurate models of environment dynamics using multistep predecessor models
- Update the value function with simulated experience generated by the environment model
- Evaluate the performance of the Dyna agent with the mitigated value hallucination
Who Needs to Know This
Researchers and engineers working on reinforcement learning and Dyna-style planning can benefit from this approach to improve the accuracy of their models and agents
Key Insight
💡 Using multistep predecessor models can help reduce the impact of model errors on Dyna agents
Share This
💡 Mitigate value hallucination in Dyna planning with multistep predecessor models
Key Takeaways
Mitigating value hallucination in Dyna planning using multistep predecessor models improves sample efficiency in reinforcement learning
Full Article
Title: Mitigating Value Hallucination in Dyna Planning via Multistep Predecessor Models
Abstract:
arXiv:2006.04363v2 Announce Type: replace-cross Abstract: Dyna-style reinforcement learning (RL) agents improve sample efficiency over model-free RL agents by updating the value function with simulated experience generated by an environment model. However, it is often difficult to learn accurate models of environment dynamics, and even small errors may result in failure of Dyna agents. In this paper, we highlight that one potential cause of that failure is bootstrapping off of the values of simu
Abstract:
arXiv:2006.04363v2 Announce Type: replace-cross Abstract: Dyna-style reinforcement learning (RL) agents improve sample efficiency over model-free RL agents by updating the value function with simulated experience generated by an environment model. However, it is often difficult to learn accurate models of environment dynamics, and even small errors may result in failure of Dyna agents. In this paper, we highlight that one potential cause of that failure is bootstrapping off of the values of simu
DeepCamp AI