Clement Gehring

clement.gehring@mila.quebec

Postdoctorate - Université de Montréal

Supervisor

Pierre-Luc Bacon

Publications

Do Transformer World Models Give Better Policy Gradients?

Michel Ma

Tianwei Ni

Clement Gehring

Pierluca D'Oro

Pierre-Luc Bacon

2024-05-01

ICML.cc/2024/Conference (poster)

doi.org

openreview.net

Bridging State and History Representations: Understanding Self-Predictive RL

Tianwei Ni

Benjamin Eysenbach

Erfan SeyedSalehi

Michel Ma

Clement Gehring

Aditya Mahajan

Pierre-Luc Bacon

Representations are at the core of all deep reinforcement learning (RL) methods for both Markov decision processes (MDPs) and partially obse… (see more)rvable Markov decision processes (POMDPs). Many representation learning methods and theoretical frameworks have been developed to understand what constitutes an effective representation. However, the relationships between these methods and the shared properties among them remain unclear. In this paper, we show that many of these seemingly distinct methods and frameworks for state and history abstractions are, in fact, based on a common idea of self-predictive abstraction. Furthermore, we provide theoretical insights into the widely adopted objectives and optimization, such as the stop-gradient technique, in learning self-predictive representations. These findings together yield a minimalist algorithm to learn self-predictive representations for states and histories. We validate our theories by applying our algorithm to standard MDPs, MDPs with distractors, and POMDPs with sparse rewards. These findings culminate in a set of preliminary guidelines for RL practitioners.

2024-01-16

ICLR.cc/2024/Conference (poster)

doi.org

openreview.net

Course Correcting Koopman Representations

Mahan Fathi

Clement Gehring

Jonathan Pilault

David Kanaa

Pierre-Luc Bacon

Ross Goroshin

2024-01-16