Haque Ishfaq

More Efficient Randomized Exploration for Reinforcement Learning via Approximate Sampling

Yixin Tan

Yu Yang

Qingfeng Lan

Jianfeng Lu

A. Rupam Mahmood

Pan Xu

2024-05-13

rl-conference.cc/RLC/2024/Conference (publié)

Provable and Practical: Efficient Exploration in Reinforcement Learning via Langevin Monte Carlo

Qingfeng Lan

Pan Xu

A. Rupam Mahmood

Anima Anandkumar

Kamyar Azizzadenesheli

We present a scalable and effective exploration strategy based on Thompson sampling for reinforcement learning (RL). One of the key shortcom… (voir plus)ings of existing Thompson sampling algorithms is the need to perform a Gaussian approximation of the posterior distribution, which is not a good surrogate in most practical settings. We instead directly sample the Q function from its posterior distribution, by using Langevin Monte Carlo, an efficient type of Markov Chain Monte Carlo (MCMC) method. Our method only needs to perform noisy gradient descent updates to learn the exact posterior distribution of the Q function, which makes our approach easy to deploy in deep RL. We provide a rigorous theoretical analysis for the proposed method and demonstrate that, in the linear Markov decision process (linear MDP) setting, it has a regret bound of

2024-01-15

ICLR.cc/2024/Conference (poster)

Offline Multitask Representation Learning for Reinforcement Learning

Raman Arora

Songtao Feng

Thanh Nguyen-Tang

Mengdi Wang

Ming Yin

We study offline multitask representation learning in reinforcement learning (RL), where a learner is provided with an offline dataset from … (voir plus)different tasks that share a common representation and is asked to learn the shared representation. We theoretically investigate offline multitask low-rank RL, and propose a new algorithm called MORL for offline multitask representation learning. Furthermore, we examine downstream RL in reward-free, offline and online scenarios, where a new task is introduced to the agent that shares the same representation as the upstream offline tasks. Our theoretical results demonstrate the benefits of using the learned representation from the upstream offline task instead of directly learning the representation of the low-rank model.

2023-12-31

Advances in Neural Information Processing Systems 37 (publié)

Randomized Exploration for Reinforcement Learning with General Value Function Approximation

Qiwen Cui

Viet Nguyen

Alex Ayoub

Zhuoran Yang

Zhaoran Wang

Lin F. Yang

2021-06-14

ArXiv (prépublication)

proceedings.mlr.press

Randomized Least Squares Policy Optimization

Zhuoran Yang

Andrei Lupu

Viet Bang Nguyen

Lewis Liu

Riashat Islam

Zhaoran Wang