Portrait de Ali Rahimi-Kalahroudi n'est pas disponible

Ali Rahimi-Kalahroudi

ali-rahimi.kalahroudi@mila.quebec
Maîtrise recherche - Université de Montréal
Superviseur⋅e principal⋅e
Sarath Chandar Anbil Parthipan
Site web
Google Scholar

Publications

Replay Buffer with Local Forgetting for Adapting to Local Environment Changes in Deep Model-Based Reinforcement Learning
Ali Rahimi-Kalahroudi
Janarthanan Rajendran
Ida Momennejad
Harm van Seijen
Sarath Chandar Anbil Parthipan
2023-01-01
CoLLAs (publié)
proceedings.mlr.press
arxiv.org
Replay Buffer With Local Forgetting for Adaptive Deep Model-Based Reinforcement Learning
Ali Rahimi-Kalahroudi
Janarthanan Rajendran
Ida Momennejad
Harm van Seijen
Sarath Chandar Anbil Parthipan
One of the key behavioral characteristics used in neuroscience to determine whether the subject of study—be it a rodent or a human—exhib… (voir plus)its model-based learning is effective adaptation to local changes in the environment. In reinforcement learning, however, recent work has shown that modern deep model-based reinforcement-learning (MBRL) methods adapt poorly to such changes. An explanation for this mismatch is that MBRL methods are typically designed with sample-efficiency on a single task in mind and the requirements for effective adaptation are substantially higher, both in terms of the learned world model and the planning routine. One particularly challenging requirement is that the learned world model has to be sufficiently accurate throughout relevant parts of the state-space. This is challenging for deep-learning-based world models due to catastrophic forgetting. And while a replay buffer can mitigate the effects of catastrophic forgetting, the traditional first-in-first-out replay buffer precludes effective adaptation due to maintaining stale data. In this work
2022-12-09
NeurIPS.cc/2022/Workshop/DeepRL (inconnu)
doi.org
openreview.net