Portrait de Wesley Chung n'est pas disponible

Wesley Chung

chungwes@mila.quebec
Doctorat - McGill
Superviseur⋅e principal⋅e
David Meger
Co-supervisor
Doina Precup
Sujets de recherche
Apprentissage en ligne
Apprentissage par renforcement
Apprentissage profond
Optimisation

Publications

Perceived Regret: Evaluating Agents in Any World
Wesley Chung
Zihan Wang
Xiao-Wen Chang
David Meger
Doina Precup
Continual reinforcement learning considers an agent receiving and learning from a stream of experience. The agent only learns about its worl… (voir plus)d through these experiences and aims to maximize its accumulated reward. A fundamental problem is to evaluate this agent using only its stream of experience, without assuming a particular structure of the world. We outline a novel approach to this problem: Define an \textit{examiner} that observes the same stream of experience and, at every timestep, computes a \textit{perceived regret}, representing the agent's suboptimality gap in value from the examiner's perspective. We empirically demonstrate how perceived regret can be a useful performance measure, applicable to arbitrary streams of experience. Theoretically, we prove that there is no universal examiner, capable of accurately assessing all agents in any world but specific modeling biases can enable success in associated environments. Further experiments show the flexibility of this approach and explore its nuances.
2026-06-15
ContinualRL @ Reinforcement Learning Conference (publié)
openreview.net
openreview.net
Parseval Regularization for Continual Reinforcement Learning
Wesley Chung
Lynn Cherif
David Meger
Doina Precup
Loss of plasticity, trainability loss, and primacy bias have been identified as issues arising when training deep neural networks on sequenc… (voir plus)es of tasks -- all referring to the increased difficulty in training on new tasks. We propose to use Parseval regularization, which maintains orthogonality of weight matrices, to preserve useful optimization properties and improve training in a continual reinforcement learning setting. We show that it provides significant benefits to RL agents on a suite of gridworld, CARL and MetaWorld tasks. We conduct comprehensive ablations to identify the source of its benefits and investigate the effect of certain metrics associated to network trainability including weight matrix rank, weight norms and policy entropy.
2024-12-10
Conference on Neural Information Processing Systems (Accept (poster))
doi.org
openreview.net
Beyond Variance Reduction: Understanding the True Impact of Baselines on Policy Optimization
Wesley Chung
Valentin Thomas
Marlos C. Machado
Nicolas Le Roux
Bandit and reinforcement learning (RL) problems can often be framed as optimization problems where the goal is to maximize average performan… (voir plus)ce while having access only to stochastic estimates of the true gradient. Traditionally, stochastic optimization theory predicts that learning dynamics are governed by the curvature of the loss function and the noise of the gradient estimates. In this paper we demonstrate that this is not the case for bandit and RL problems. To allow our analysis to be interpreted in light of multi-step MDPs, we focus on techniques derived from stochastic optimization principles (e.g., natural policy gradient and EXP3) and we show that some standard assumptions from optimization theory are violated in these problems. We present theoretical results showing that, at least for bandit problems, curvature and noise are not sufficient to explain the learning dynamics and that seemingly innocuous choices like the baseline can determine whether an algorithm converges. These theoretical findings match our empirical evaluation, which we extend to multi-state MDPs.
2021-07-17
International Conference on Machine Learning (inconnu)
doi.org
proceedings.mlr.press