Portrait de Donna Vakalis

Donna Vakalis

Collaborateur·rice alumni - UdeM
Superviseur⋅e principal⋅e
Yoshua Bengio
Co-supervisor
David Rolnick
Sujets de recherche
Apprentissage par renforcement
Apprentissage profond
Réseaux de neurones en graphes
Site web
Google Scholar
GitHub

Publications

In-Context Reinforcement Learning through Bayesian Fusion of Context and Value Prior
Anaïs Berkes
Vincent Taboga
Donna Vakalis
David Rolnick
Yoshua Bengio
In-context reinforcement learning (ICRL) promises fast adaptation to unseen environments without parameter updates, but current methods eith… (voir plus)er cannot improve beyond the training distribution or require near-optimal data, limiting practical adoption. We introduce SPICE, a Bayesian ICRL method that learns a prior over Q-values via deep ensemble and updates this prior at test-time using in-context information through Bayesian updates. To recover from poor priors resulting from training on sub-optimal data, our online inference follows an Upper-Confidence Bound rule that favours exploration and adaptation. We prove that SPICE achieves regret-optimal behaviour in both stochastic bandits and finite-horizon MDPs, even when pretrained only on suboptimal trajectories. We validate these findings empirically across bandit and control benchmarks. SPICE achieves near-optimal decisions on unseen tasks, substantially reduces regret compared to prior ICRL and meta-RL approaches while rapidly adapting to unseen tasks and remaining robust under distribution shift.
2025-12-31
arXiv (prépublication)
doi.org
arxiv.org
A HOT Dataset: 150,000 Buildings for HVAC Operations Transfer Research
Anaïs Berkes
Yoshua Bengio
David Rolnick
Donna Vakalis
2025-11-10
Proceedings of the 12th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation (publié)
doi.org
A HOT Dataset: 150,000 Buildings for HVAC Operations Transfer Research
Anaïs Berkes
Yoshua Bengio
David Rolnick
Donna Vakalis
About 12% of global energy consumption is attributable to heating, ventilation, and air conditioning (HVAC) systems in buildings [11]. Machi… (voir plus)ne learning-based intelligent HVAC control offers significant energy efficiency potential, but progress is constrained by limited data for training and evaluating performance across different kinds of buildings. Existing datasets primarily target energy prediction rather than control applications, forcing studies to rely on limited building sets or single-variable perturbations that fail to capture real-world complexity. We present HOT (HVAC Operations Transfer), the first large-scale open-source dataset purpose-built for research into transfer learning in building control. HOT contains 159,744 unique building-weather combinations with systematic variations across envelope properties, occupancy patterns, and climate conditions spanning all 19 ASHRAE climate zones across 76 global locations. We formalise a comprehensive similarity-based framework with quantitative metrics for assessing transfer feasibility between source and target buildings across multiple context dimensions. Our key contributions: (1) a large-scale, open dataset and tooling enabling systematic, multi-variable transfer studies across 19 climate zones; (2) a quantitative similarity framework spanning geometry, thermal, climate, and function; and (3) zero-shot climate transfer experiments showing why realistic context variation matters for HVAC control.
2025-11-10
Proceedings of the 12th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation (publié)
doi.org
HVAC-SPICE: Value-Uncertainty In-Context RL with Thompson Sampling for Zero-Shot HVAC Control
Anaïs Berkes
Vincent Taboga
Donna Vakalis
David Rolnick
Yoshua Bengio
Urban buildings consume 40\% of global energy, yet most rely on inefficient rule-based HVAC systems due to the impracticality of deploying a… (voir plus)dvanced controllers across diverse building stock. In-context reinforcement learning (ICRL) offers promise for rapid deployment without per-building training, but standard supervised learning objectives that maximise likelihood of training actions inherit behaviour-policy bias and provide weak exploration under the distribution shifts common when transferring across buildings and climates. We present SPICE (Sampling Policies In-Context with Ensemble uncertainty), a novel ICRL method specifically designed for zero-shot building control that addresses these fundamental limitations. SPICE introduces two key methodological innovations: (i) a propensity-corrected, return-aware training objective that prioritises high-advantage, high-uncertainty actions to enable improvement beyond suboptimal training demonstrations, and (ii) lightweight value ensembles with randomised priors that provide explicit uncertainty estimates for principled episode-level Thompson sampling. At deployment, SPICE samples one value head per episode and acts greedily, resulting in temporally coherent exploration without test-time gradients or building-specific models. We establish a comprehensive experimental protocol using the HOT dataset to evaluate SPICE across diverse building types and climate zones, focusing on the energy efficiency, occupant comfort, and zero-shot transfer capabilities that are critical for urban-scale deployment.
2025-09-29
NeurIPS.cc/2025/Workshop/UrbanAI (poster)
openreview.net
openreview.net
Graph Dreamer: Temporal Graph World Models for Sample-Efficient and Generalisable Reinforcement Learning
Anaïs Berkes
Donna Vakalis
Yoshua Bengio
David Rolnick
2025-09-21
NeurIPS.cc/2025/Workshop/WiML (publié)
openreview.net
openreview.net
HVAC-GRACE: Transferable Building Control via Heterogeneous Graph Neural Network Policies
Anaïs Berkes
Donna Vakalis
David Rolnick
Yoshua Bengio
Buildings consume 40% of global energy, with HVAC systems responsible for up to half of that demand. As energy use grows, optimizing HVAC ef… (voir plus)ficiency is critical to meeting climate goals. While reinforcement learning (RL) offers a promising alternative to rule-based control, real-world adoption is limited by poor sample efficiency and generalisation. We introduce HVAC-GRACE, a graph-based RL framework that models buildings as heterogeneous graphs and integrates spatial message passing directly into temporal GRU gates. This enables each zone to learn control actions informed by both its own history and its structural context. Our architecture supports zero-shot transfer by learning topology-agnostic functions—but initial experiments reveal that this benefit depends on sufficient conditioned zone connectivity to maintain gradient flow. These findings highlight both the promise and the architectural requirements of scalable, transferable RL for building control
2025-06-30
ICML.cc/2025/Workshop/CO-BUILD (poster)
openreview.net
openreview.net