Portrait de Pierluca D'Oro

Pierluca D'Oro

Membre affilié
Chercheur scientifique, Meta
Sujets de recherche
Apprentissage par renforcement
Grands modèles de langage (LLM)
IA centrée sur l'humain
IAG (Intelligence Artificielle Générale)
Site web
Google Scholar
LinkedIn
X

Publications

The Primacy Bias in Deep Reinforcement Learning
Evgenii Nikishin
Max Schwarzer
Pierluca D'Oro
Pierre-Luc Bacon
Aaron Courville
This work identifies a common flaw of deep reinforcement learning (RL) algorithms: a tendency to rely on early interactions and ignore usefu… (voir plus)l evidence encountered later. Because of training on progressively growing datasets, deep RL agents incur a risk of overfitting to earlier experiences, negatively affecting the rest of the learning process. Inspired by cognitive science, we refer to this effect as the primacy bias. Through a series of experiments, we dissect the algorithmic aspects of deep RL that exacerbate this bias. We then propose a simple yet generally-applicable mechanism that tackles the primacy bias by periodically resetting a part of the agent. We apply this mechanism to algorithms in both discrete (Atari 100k) and continuous action (DeepMind Control Suite) domains, consistently improving their performance.
2022-06-27
Proceedings of the 39th International Conference on Machine Learning (publié)
doi.org
proceedings.mlr.press
Long-Term Credit Assignment via Model-based Temporal Shortcuts
Michel Ma
Pierluca D'Oro
Yoshua Bengio
Pierre-Luc Bacon
2021-12-12
NeurIPS.cc/2021/Workshop/DeepRL (inconnu)
openreview.net
openreview.net