Portrait de Guangyuan Wang n'est pas disponible

Guangyuan Wang

guangyuan.wang@mila.quebec
Stagiaire de recherche - McGill
Superviseur⋅e principal⋅e
Doina Precup
Sujets de recherche
Apprentissage par renforcement
Apprentissage profond
Théorie de l'apprentissage automatique

Publications

Learning Where to Constrain: Meta-Learned Trust Allocation for Mathematical Reasoning
Guangyuan Wang
Padideh Nouri
Doina Precup
Long mathematical solutions expose a weakness in uniform policy-update constraints. Early token changes affect a long suffix, while late cha… (voir plus)nges affect fewer future states. Prefix divergence-based methods like CPPO encode this asymmetry with a fixed position schedule and prefix budget. We introduce Meta-TA, a bilevel method for learning both choices from post-update performance on disjoint prompts. A direct one-step design fails when behavior and current policies match because the constraint starts inactive. We prove the resulting hypergradient equals zero. A detached warm-up creates controlled policy drift. A soft-gated step then supplies the meta-gradient. On Qwen3-8B-Base, five frozen-retraining seeds raise mean AIME24/25/26 Avg@16 from 29.01 for CPPO to 31.95 for Meta-TA with closely matched realized KL. At a 16k response cap, Meta-TA lowers normalized reasoning effective horizon from 0.475 to 0.430 and importance-ratio variance from 0.0534 to 0.0475. These results link adaptive trust allocation to higher accuracy and more localized updates in long-horizon mathematical reasoning.
2026-10-03
MATH-AI @ Neural Information Processing Systems (poster)
openreview.net
openreview.net
Phases of Muon: When Muon Eclipses SignSGD
Elliot Paquette
Noah Marshall
Lucas Benigni
Guangyuan Wang
Atish Agarwala
Courtney Paquette
Recently, Muon and related spectral optimizers have demonstrated strong empirical performance as scalable stochastic methods, often outperfo… (voir plus)rming Adam. Yet their behaviour remains poorly understood. We analyze stochastic spectral optimizers, including Muon, on a high-dimensional matrix-valued least squares problem. We derive explicit deterministic dynamics that provide a tractable framework for studying learning behaviour with a focus on (stochastic) SignSVD, which Muon approximates, and (stochastic) SignSGD, the latter serving as a proxy for Adam. Our analysis shows that for large batch size, SignSVD performs a square-root preconditioning with respect to the data covariance spectrum, while for small batch size smaller eigenmodes behave like SGD, slowing down convergence. We contrast with SignSGD which for generic covariance performs no preconditioning and has no transition, leading to different optimal learning rates and convergence characteristics. The two methods match up to a constant factor with isotropic data, but behave differently with anisotropic data. An analysis of a power law covariance model with data exponent
2026-05-09
arXiv (prépublication)
doi.org
arxiv.org
Langevin Soft Actor-Critic: Efficient Exploration Through Uncertainty-Driven Critic Learning
Haque Ishfaq
Guangyuan Wang
Sami Nur Islam
Doina Precup
Existing actor-critic algorithms, which are popular for continuous control reinforcement learning (RL) tasks, suffer from poor sample effici… (voir plus)ency due to lack of principled exploration mechanism within them. Motivated by the success of Thompson sampling for efficient exploration in RL, we propose a novel model-free RL algorithm, Langevin Soft Actor Critic (LSAC), which prioritizes enhancing critic learning through uncertainty estimation over policy optimization. LSAC employs three key innovations: approximate Thompson sampling through distributional Langevin Monte Carlo (LMC) based
2025-04-22
International Conference on Learning Representations (Accept (Poster))
doi.org
openreview.net