Publications

Addressing Sparse-Rewards in RL with Scalable Hierarchical Novel Eigen Options
Priyesh Vijayan
Élodie Coté-Gauthier
Mathieu Reymond
Sarath Chandar
Doina Precup
Isabeau Prémont-Schwarz
Temporally extended exploration via graph Laplacian-based options is a promising approach to sparse-reward reinforcement learning (RL), but … (voir plus)existing methods either do not explicitly target novelty or fail to scale to pixel-based domains under function approximation. Novel Exploration via Orthogonality (NEO) addresses the first issue by constructing options that navigate from highly visited regions toward less visited ones, yet prior results were limited to settings where exact eigenvectors can be computed. We present a scalable extension of NEO to pixel-based domains, built on three contributions. First, we use a novelty-weighted continuous Laplacian graph-drawing objective, which enables RL with continuous observations. Second, we embed the resulting eigen-potential options within a hierarchical reinforcement learning framework, enabling coherent temporally extended behavior. Third, we observe that learned eigen-potential rewards are directional but locally unreliable under online approximation; we therefore augment each option reward with a novelty bonus, a novel design idea that proves essential for stabilizing option learning while preserving novelty-directed exploration. Together, these contributions yield stronger and more persistent exploration, enabling longer option rollouts and better access to hard-to-reach novel states. Empirically, our method significantly outperforms both the prior scalable Laplacian-option baseline and a direct extension of NEO on sparse-reward benchmarks under a fixed budget. On Montezuma's Revenge, our best variant achieves approximately 1.8x higher return than both baselines. On Venture, both baselines yield returns near zero, whereas our method achieves a return of 1135. Across seven hard ProcGen games, our method achieves approximately 3.5x and 5.6x higher aggregate normalized return than the two baselines, respectively.
2026-07-20
RLBRew @ Reinforcement Learning Conference (publié)
openreview.net
openreview.net
Enhancing Psychiatry Training Using an Agentic AI Simulated Consultation Tool: Prospective Cohort Study
Alice Rueda
Huda F Al-Shamali
Zack Cote
Niloy Roy
Reinhard Janssen‐Aguilar
Jithin Joseph
Bazen Gashaw Teferra
Mohammad Amin Kamaleddin
Lisa Burback
Olga Winkler
Bill Kapralos
Andrei Torres
Divya Sharma
Sridhar Krishnan
Guillaume Dumas
Andrew J. Greenshaw
Alexandre Hudon
Sanjeev Sockalingam
Yanbo Zhang
Adam Dubrowski … (voir 1 de plus)
Venkat Bhat
Background: Canadian psychiatry residents must demonstrate consultation competency, assessed using the standardized assessment of a clinical… (voir plus) encounter report (STACER). However, opportunities to practice these skills and receive constructive assessment remain limited in clinical settings. Objective: This study aimed to evaluate the technical feasibility of an agentic AI system designed to support psychiatry residents' consultation competence through simulated patient encounters with a patient agent and structured feedback from a rater agent. Methods: We conducted a two-phase technical feasibility prospective single-arm cohort study of the STACER Agentic System, a large language model-based platform integrating a patient agent and a rater agent. Phase 1 involved automated evaluation of the patient agent using a psychiatrist agent across 227 synthetic major depressive disorder cases. Performance was assessed using DeepEval metrics (correctness, clarity, medical faithfulness, turn relevance, and role adherence) with descriptive statistics and 95% CIs. Phase 2 involved a preliminary user study with 14 convenience-sampled participants: a total of 5 members of the clinical research team and 9 psychiatry residents from the University of Alberta. Participants completed simulated diagnostic interviews and case presentations. Performance was evaluated using STACER-based scoring by the rater agent and 2 psychiatrists. Interrater reliability was assessed using intraclass correlation coefficients (α=.05). Participants rated realism, behavioral consistency, psychiatric nuance, and feedback utility using Likert scales and free-text answers. Results: The patient agent demonstrated high behavioral (51/56, 91.07%) and symptom fidelity (105/110, 95.45%), with strong automated performance (medical faithfulness mean 0.99, 95% CI 0.99-1.00; turn relevance 0.99, 95% CI 0.986-0.992). Participants rated simulations as psychiatrically plausible and diagnostically useful, particularly for depressive symptom representation, although rapport building was moderate (mean 2.78, SD 1.56 to mean 3.00, SD 1.41, out of 5.00) due to limited nonverbal cues. The rater agent generated structured STACER-aligned feedback with high intrarater consistency, especially at the section subtotal level. Interrater reliability with psychiatrists was poor at the item level (intraclass correlation coefficient range=0.25-0.49) but improved to good-to-excellent agreement at the section level for psychiatry resident sessions (intraclass correlation coefficient range=0.89-0.93). The rater agent's scores fell between those of the 2 psychiatrists for the clinical research team and were lower than both human raters for psychiatry residents. Conclusions: The STACER Agentic System demonstrates the technical feasibility of using agentic AI to simulate psychiatric consultations and deliver STACER-aligned formative feedback. By combining adaptive multiturn psychiatric simulation with competency-based evaluation, it shows promise in supporting cognitive aspects of consultation, though it remains limited in facilitating relational skills such as rapport building. These findings suggest agentic AI could expand scalable, low-risk opportunities for deliberate practice and formative feedback in competency-based psychiatric education. Further controlled studies are needed to evaluate educational effectiveness and integration into residency training.
2026-07-20
JMIR Medical Education (publié)
doi.org
Imagining Goals for Policy Generalization with Geometric Horizon Models
Daniel Lawson
Dhruv Sreenivas
Adriana Hugessen
Khimya Khetarpal
Glen Berseth
Training from offline data has allowed for substantial progress in domains such as robotics, leading to general-purpose policies that can be… (voir plus) easily applied zero-shot or efficiently finetuned for downstream tasks. However, training policies on offline data can lead to poor generalization, both due to the choice of modeling objective and from learning from a static dataset. In this work, we focus on the challenging task of zero-shot goal generalization, where a policy is evaluated on unseen tasks that require reusing its existing knowledge (compositional generalization). An avenue for improving a policy's generalization is through generating new experience through world models; however, such generation has proven difficult for longer horizons. Thus, to alleviate this issue, we propose TD-Aug, sampling from a geometric horizon model, which allows for directly imagining novel outcomes that can be achieved through composing existing knowledge. We demonstrate that training on these future outcomes as goals for goal-conditioned BC and offline RL policies improves generalization in stitching-based OGBench tasks.
2026-07-20
RLBRew @ Reinforcement Learning Conference (publié)
openreview.net
openreview.net
AInimation: Animating from Prompt to AI-Generated Responses
Jiaqi Wu
Damien Masson
We explore the use of animated transitions between a prompt and an AI-generated response. After reviewing 800 examples of prompts and respon… (voir plus)ses, we devise a taxonomy of animated transitions for multimodal text- and image-generative models. The proposed animations include translating and morphing elements of the prompt to their final location in the response; highlighting modifications such as fixed typos; overlaying structural requirements to verify them; and displaying how a model understands references. A study shows that adding animated transitions helps users review the response: participants performed 43% better at locating elements in the response; 153% better at identifying changes; and 20% better at verifying the prompt was correctly interpreted. Our work applies to all software that integrates AI and shows that well-crafted, slower animations are preferable to instant AI responses.
2026-07-19
arXiv (prépublication)
doi.org
arxiv.org
DA-MergeLoRA: Hypernetwork-Based LoRA Merging for Few-Shot Test-Time Domain Adaptation
Siobhan Reid
Zhixiang Chi
Li Gu
Omid Reza Heidari
Ziqiang Wang
Yang Wang
Few-shot Test-Time Domain Adaptation (FSTT-DA) seeks to adapt models to novel domains using only a handful of unlabeled target samples. This… (voir plus) setting is more realistic than typical domain adaptation setups, which assume access to target data during source training. However, prior FSTT-DA approaches fail to effectively leverage source domain-specific knowledge, relying on shallow batch normalization updates, prompt-based methods that treat the model as a black box, or ensembling strategies that do not capture cross-domain relationships. To address these limitations, we introduce a new FSTT-DA framework that integrates LoRA fine-tuning with model merging. In our approach, separate LoRA modules are fine-tuned on CLIP's vision encoder for each source domain. Since LoRA modifies only a small fraction of the model's parameters, it retains the base model's generalized knowledge while internally learning domain-specific features. To adapt the learned knowledge to a specific target domain, we propose a hypernetwork trained via meta-learning that generates per-column merging factors to combine LoRA modules. Given a small batch of target images, the hypernetwork produces merging weights that fuse source LoRA modules into a single adapted representation. Our results demonstrate state-of-the-art performance across various domain adaptation datasets. Our code is publicly available at https://github.com/nahbois4321/DA-MergeLoRA.
2026-07-19
arXiv (prépublication)
doi.org
arxiv.org
Insuring AI: Incentivising Safe and Secure Deployment
Agni Orfanoudaki
Agni Orfanoudaki
Carsten Maple
Matthew Wicker
Yoshua Bengio
Kwok-Yan Lam
Marcin Detyniecki
Lukasz Szpruch
2026-07-19
Harvard Data Science Review (publié)
doi.org
Interpreting Physics in Video World Models: Toward Safe Embodied AI
Sonia Joseph
Quentin Garrido
Randall Balestriero
Matthew Kowal
Thomas Fel
Shahab Bakhtiari
Blake Aaron Richards
Michael Rabbat
Physical AI agents acting in the real world fail differently from language models: a misjudgment of trajectory, force, or contact can have i… (voir plus)mmediate and potentially irreversible consequences. As video world models are increasingly deployed as the perception and dynamics layer of such agents, understanding what physical structure they actually represent internally be- comes a precondition for trustworthy deployment. Yet today these models are largely studied as black boxes, evaluated only through behavioral benchmarks. We argue that this should change: world models can be opened up, and the latent variables they encode can be measured, interpreted, and eventually controlled. We take a first step in this direction across two state-of-the-art video encoders (V-JEPA 2 and VideoMAE-v2), using layerwise probing, subspace geometry, patch-level decoding, and targeted attention ablations. We identify a sharp intermediate-depth transition we call the Physics Emergence Zone, at which physical variables become accessible. Decomposing motion into explicit variables, we find that scalar quantities such as speed and acceleration are available from early layers onwards, whereas motion direction becomes accessible only at the Physics Emergence Zone, encoded as a high-dimensional circular population code that requires coordinated multi-feature intervention to steer. These findings argue against compact, reusable latent physics state and in favor of distributed, task-specific representations supported by a shared local-attention circuit. We discuss implications for safe deploy- ment of video world models in embodied physical AI, including why intermediate-layer features, multi-feature monitors, and the local-attention circuit at the Physics Emergence Zone are natural targets for runtime verification.
2026-07-19
SPAI @ International Joint Conference on Artificial Intelligence - European Conference on Artificial Intelligence (publié)
openreview.net
openreview.net
PPL-Factory: Task-Aware and Budget-Aware Data Selection from Language Modeling to Reasoning
Hang Zhang
Warren J. Gross
Not all training samples contribute equally to large language model fine-tuning. Selecting informative training samples can reduce the compu… (voir plus)tational cost while preserving downstream performance. Many existing data selection methods rely on indirect heuristics, such as data quality, diversity or reasoning trace length. However, the effectiveness of these fixed criteria is task-dependent and difficult to generalize across diverse downstream tasks. Perplexity-based data selection provides a simple and model-aware solution to estimate the sample difficulty, but existing approaches typically score the entire training sequence and ignore the difference in learning objectives of language modeling and reasoning tasks. In this paper, we propose PPL-Factory, a simple and interpretable data selection framework that combines task-aware perplexity-based scores and data budget-aware selection criteria. Experiments on GSM8K demonstrate that PPL-Factory outperforms other state-of-the-art data selection methods using only
2026-07-19
arXiv (prépublication)
doi.org
arxiv.org
Search-on-Graph-R1: Training Large Language Models to Search Knowledge Graphs with Reinforcement Learning
Jia Ao Sun
Yu Hao
Fengran Mo
Zhan Su
Yuchen Hui
Bang Liu
Jian-Yun Nie
Knowledge graph question answering (KGQA) requires navigating from topic entities to an answer several relations away. Recent methods prompt… (voir plus) a frontier LLM to explore the graph through a retrieval tool, but their reliance on frontier-scale inference makes them costly to deploy. We present Search-on-Graph-R1 (\sogrone{}), which internalizes this navigation into a compact 8B model through supervised fine-tuning (SFT) followed by reinforcement learning (RL). Our central idea is to scaffold a frontier teacher with each question's gold SPARQL query, so the teacher traverses a known answer-bearing path with a live \texttt{Search} tool rather than having to discover the path itself. Since every call executes against a live Freebase server, the resulting trajectories are grounded in the knowledge graph by construction. On WebQSP, CWQ, and GrailQA, \sogrone{} at 8B surpasses every frozen frontier-LLM system in our comparison and posts the strongest results on CWQ of any system we compare against. It does so using no auxiliary module at inference and no LLM judge during training. Isolating each training stage shows that SFT and RL contribute complementary gains, our approach transfers across model families, and RL learns to reach answers in fewer \texttt{Search} calls than its SFT initialization.
2026-07-19
arXiv (prépublication)
doi.org
arxiv.org
Equivalent Numeric Approved Identity of Passport and Immigration Papers -- The Next Generation Identification Program (NGI) [A Short Introduction]
Nonvikan Karl-Augustt Alahassa
Bidossessi R.U. Alahassa
Julie Carrier
Joel TOSSA
Wilfrid Gangbo
Jörg Matthes
Cyriaque Atindogbé
Marlène Frigon
Christiane Rousseau
Maciej Augustyniak
Dimitrios Koukopoulos
Samuel Bassetto
Foutse Khomh
J. B. Chabi Orou
Jérôme Théau
Damien Échevin
Raphael R. Kelani
David Haziza
Emmanuel Stip
Mylène Bédard … (voir 8 de plus)
Élise Vandomme
Mouloud-Beallah Belbahri
Bakary Manga
Daniel F. Nadeau
Suljo Linic
Bruno Rémillard
Victor M. Panaretos
Nathalie Lacelle
We express an interest to Equivalent Numeric Approved Identity of Passport and Immigration Papers. [We are making a submission to the Journa… (voir plus)l of Banking and Financial Technology].
2026-07-17
Zenodo (inconnu)
doi.org
Isotonic Conformal Prediction
Daniel Bensimon
Sean Xiang Yu
Eric D. Kolaczyk
Archer Y. Yang
A point prediction that is well calibrated on average can still be systematically biased conditional on its own value, undermining its use i… (voir plus)n downstream decision-making. We consider two objectives for reliable uncertainty quantification: self-calibration, requiring a point prediction to be unbiased conditional on its own value, and prediction-conditional validity, requiring a prediction interval to attain nominal coverage conditional on the prediction. Self-Calibrating Conformal Prediction (SC-CP) attains both objectives exactly in finite samples, but requires refitting its calibrator for every candidate outcome, which is computationally prohibitive for continuous outcomes. We propose Isotonic Conformal Prediction (ICP), a framework that decouples calibration from prediction-set construction by fitting a single isotonic recalibration map and constructing prediction intervals within strata of similar recalibrated predictions. Within this framework we develop two procedures. Split Isotonic Conformal Prediction (SICP) attains prediction-conditional validity in finite samples and self-calibration asymptotically, at the computational cost of split conformal prediction. Transductive Isotonic Conformal Prediction (TICP) attains both objectives exactly in finite samples through a per-test-point inner loop that avoids refitting the isotonic calibrator. On synthetic heteroscedastic regression problems and a real-world healthcare-utilization dataset, both procedures match the coverage of SC-CP at substantially lower computational cost.
2026-07-17
arXiv (prépublication)
doi.org
arxiv.org
Memorandum, Notes on Visitations and Ongoing Projects in Canada (Version -1)
Nonvikan Karl-Augustt Alahassa
Bidossessi R.U. Alahassa
Julie Carrier
Nathalie Lacelle
Samuel Bassetto
Foutse Khomh
Leonard Wantchekon
Mylène Bédard
Emmanuel Stip
Bruno Rémillard
Suljo Linic
J. Tossa
Bakary Manga
Jérôme Théau
Victor M. Panaretos
David Haziza
Daniel F. Nadeau
Marlène Frigon
Christiane Rousseau
Maciej Augustyniak … (voir 8 de plus)
Dimitrios Koukoulopoulos
Vandomme, Élise
Gangbo, Wilfrid
Kelani, Raphael
Chabi Orou, Jean Bio
ATINDOGBE, Comlan Cyriaque
Wilfrid Gangbo
Raphael Kelani
We are on the Refugee Claimant list in Canada. We would like to express few notes of Memorandum.
2026-07-17
Zenodo (inconnu)
doi.org