Portrait de Pierluca D'Oro

Pierluca D'Oro

Membre affilié
Chercheur scientifique, Meta
Sujets de recherche
Apprentissage par renforcement
Grands modèles de langage (LLM)
IA centrée sur l'humain
IAG (Intelligence Artificielle Générale)
Site web
Google Scholar
LinkedIn
X

Publications

Unleashing The Potential of Data Sharing in Ensemble Deep Reinforcement Learning
Zhixuan Lin
Pierluca D'Oro
Evgenii Nikishin
Aaron Courville
This work studies a crucial but often overlooked element of ensemble methods in deep reinforcement learning: data sharing between ensemble m… (voir plus)embers. We show that data sharing enables peer learning, a powerful learning process in which individual agents learn from each other's experience to significantly improve their performance. When given access to the experience of other ensemble members, even the worst agent can match or outperform the previously best agent, triggering a virtuous circle. However, we show that peer learning can be unstable when the agents' ability to learn is impaired due to overtraining on early data. We thus employ the recently proposed solution of periodic resets and show that it ensures effective peer learning. We perform extensive experiments on continuous control tasks from both dense states and pixels to demonstrate the strong effect of peer learning and its interaction with resets.
2022-12-08
NeurIPS.cc/2022/Workshop/DeepRL (inconnu)
openreview.net
openreview.net
The Primacy Bias in Deep Reinforcement Learning
Evgenii Nikishin
Max Schwarzer
Pierluca D'Oro
Pierre-Luc Bacon
Aaron Courville
This work identifies a common flaw of deep reinforcement learning (RL) algorithms: a tendency to rely on early interactions and ignore usefu… (voir plus)l evidence encountered later. Because of training on progressively growing datasets, deep RL agents incur a risk of overfitting to earlier experiences, negatively affecting the rest of the learning process. Inspired by cognitive science, we refer to this effect as the primacy bias. Through a series of experiments, we dissect the algorithmic aspects of deep RL that exacerbate this bias. We then propose a simple yet generally-applicable mechanism that tackles the primacy bias by periodically resetting a part of the agent. We apply this mechanism to algorithms in both discrete (Atari 100k) and continuous action (DeepMind Control Suite) domains, consistently improving their performance.
2022-06-27
Proceedings of the 39th International Conference on Machine Learning (publié)
doi.org
proceedings.mlr.press
Long-Term Credit Assignment via Model-based Temporal Shortcuts
Michel Ma
Pierluca D'Oro
Yoshua Bengio
Pierre-Luc Bacon
2021-12-12
NeurIPS.cc/2021/Workshop/DeepRL (inconnu)
openreview.net
openreview.net