Portrait of Weikai Wang is unavailable

Weikai Wang

weikai.wang@mila.quebec
PhD - HEC Montréal
Supervisor
Erick Delage
Research Topics
Optimization
Reinforcement Learning

Publications

Dynamic Programming for Value-at-risk Markov Decision Processes in Continuous Spaces
Xian Chen
Erick Delage
Weikai Wang
Value-at-Risk (VaR) is a widely adopted quantile-based risk measure in practice, yet incorporating VaR objectives into Markov decision proce… (see more)sses (VaR MDPs) remains a longstanding challenge. Recent work has derived dynamic programming equations for VaR MDPs in finite state and action spaces but left the case of more general spaces unresolved. This paper establishes, for the first time, the validity of these equations in continuous spaces, for both finite- and infinite-horizon settings. Such cases naturally arise in many real-world dynamic decision-making problems. As an application, we study an optimal execution problem with a VaR objective and demonstrate that our framework enables the characterization of optimal dynamic strategies, resolving an open question in the field.
2026-10-16
arXiv (preprint)
doi.org
arxiv.org
Online Policy Evaluation for MDPs with Dynamic UBSR Measures
Weikai Wang
Erick Delage
Developing efficient function-approximation methods for policy evaluation is a fundamental challenge in risk-aware reinforcement learning. E… (see more)xisting approaches either focus on restrictive classes of risk measures or rely on access to a simulator, limiting their applicability in fully online settings. In this work, we propose computationally efficient online learning algorithms for policy evaluation in Markov decision processes (MDPs) with dynamic utility-based shortfall risk (UBSR) measures under linear function approximation. Specifically, we introduce the UBSR-TD algorithm, establish conditions under which it converges almost surely, and develop several variants designed to accelerate convergence. Our formulation shows that existing policy evaluation algorithms for risk-neutral MDPs can be readily adapted to dynamic UBSR settings by incorporating a loss function into the temporal-difference error. Numerical experiments support our theoretical findings, and an application to a perishable inventory management problem with shelf-life uncertainty demonstrates the practical effectiveness of the proposed methods.
2026-07-24
arXiv (preprint)
doi.org
arxiv.org
Planning and Learning in Average Risk-aware MDPs
Weikai Wang
Erick Delage
For continuing tasks, average cost Markov decision processes have well-documented value and can be solved using efficient algorithms. Howeve… (see more)r, it explicitly assumes that the agent is risk-neutral. In this work, we extend risk-neutral algorithms to accommodate the more general class of dynamic risk measures. Specifically, we propose a relative value iteration (RVI) algorithm for planning and design two model-free Q-learning algorithms, namely a generic algorithm based on the multi-level Monte Carlo (MLMC) method, and an off-policy algorithm dedicated to utility-based shortfall risk measures. Both the RVI and MLMC-based Q-learning algorithms are proven to converge to optimality. Numerical experiments validate our analysis, confirm empirically the convergence of the off-policy algorithm, and demonstrate that our approach enables the identification of policies that are finely tuned to the intricate risk-awareness of the agent that they serve.
2025-12-02
Conference on Neural Information Processing Systems (Accept (poster))
doi.org
openreview.net