Portrait of Pierluca D'Oro

Pierluca D'Oro

Affiliate Member
Research Scientist, Meta
Research Topics
AGI (Artificial General Intelligence)
Human-Centered AI
Large Language Models (LLM)
Reinforcement Learning
Website
Google Scholar
LinkedIn
X

Publications

The Primacy Bias in Deep Reinforcement Learning
Evgenii Nikishin
Max Schwarzer
Pierluca D'Oro
Pierre-Luc Bacon
Aaron Courville
This work identifies a common flaw of deep reinforcement learning (RL) algorithms: a tendency to rely on early interactions and ignore usefu… (see more)l evidence encountered later. Because of training on progressively growing datasets, deep RL agents incur a risk of overfitting to earlier experiences, negatively affecting the rest of the learning process. Inspired by cognitive science, we refer to this effect as the primacy bias. Through a series of experiments, we dissect the algorithmic aspects of deep RL that exacerbate this bias. We then propose a simple yet generally-applicable mechanism that tackles the primacy bias by periodically resetting a part of the agent. We apply this mechanism to algorithms in both discrete (Atari 100k) and continuous action (DeepMind Control Suite) domains, consistently improving their performance.
2022-06-27
Proceedings of the 39th International Conference on Machine Learning (published)
doi.org
proceedings.mlr.press
Long-Term Credit Assignment via Model-based Temporal Shortcuts
Michel Ma
Pierluca D'Oro
Yoshua Bengio
Pierre-Luc Bacon
2021-12-12
NeurIPS.cc/2021/Workshop/DeepRL (unknown)
openreview.net
openreview.net