Portrait of Zijing Wu is unavailable

Zijing Wu

zijing.wu@mila.quebec
PhD - McGill University
Supervisor
Paul Masset
Co-supervisor
Doina Precup
Research Topics
Computational Neuroscience
NeuroAI
Neuroscience
Reinforcement Learning
Website
Google Scholar
Bluesky
GitHub

Publications

QD-Learning for Continual Reinforcement Learning
Zijing Wu
Doina Precup
Paul Masset
Nishanth Anand
In continual reinforcement learning (CRL), an agent continually adapts to an unbroken stream of experience by balancing two conflicting obje… (see more)ctives: stability and plasticity. To balance these two objectives, we introduce QD-learning, a new class of algorithms rooted in KL-regularized RL. QD-learning algorithms combine \textit{an adaptive default policy} with \textit{a soft-Q policy} to determine the agent's behaviour at each time step. Similar to habit learning in the brain, the adaptive default policy is updated to mimic the agent's overall behaviour, while the soft-Q policy is updated using TD-error along with a regularizing term based on the KL divergence between the default and the agent's current policies. Our paradigm, QD-learning, encompasses capacity-constrained RL and existing KL-regularized RL approaches as special cases while remaining general. We present three instantiations of QD-learning in dynamic programming and sample-based settings and analyze their theoretical properties via policy-improvement theorems. Additionally, we empirically evaluate our algorithms on both single-task and continual RL settings using a tabular chain MDP. The results show that QD-learning clearly outperforms baselines, particularly in environments with a large number of actions.
2026-06-15
ContinualRL @ Reinforcement Learning Conference (published)
openreview.net
openreview.net