Publications

ECUAS: A family of metrics for principled evaluation of uncertainty-augmented systems
Lautaro Estienne
Erik Ernst
Matias Vera
Pablo Piantanida
LUCIANA FERRER
In high-stakes automated decision-making, access to predictive uncertainty is essential for enabling users -- human or downstream systems --… (voir plus) to accept or reject predictions based on application-specific cost trade-offs. Such uncertainty-augmented (UA) systems -- i.e., systems that output both predictions and uncertainty scores -- are currently being assessed in the literature in a variety of ways, using separate metrics to evaluate the predictions and the uncertainty scores, setting a cost function with a fixed rejection cost or integrating over a coverage-risk curve. We argue that these evaluation approaches are inadequate for assessing overall performance of the UA system for decision making under uncertainty and propose a novel family of metrics, ECUAS
2026-12-30
OpenReview (publié)
openreview.net
openreview.net
QueStER: Query Specification for Generative Keyword-Based Retrieval
Arthur SATOUF
Yuxuan Zong
Habiboulaye Amadou Boubacar
Pablo Piantanida
Benjamin Piwowarski
Generative retrieval (GR) differs from the traditional index–then–retrieve pipeline by storing relevance in model parameters and generat… (voir plus)ing retrieval cues directly from the query, but it can be brittle out of domain and expensive to scale. We introduce QueStER (QUEry SpecificaTion for gEnerative Keyword-Based Retrieval), which bridges GR and query reformulation by learning to generate explicit keyword-based search specifications. Given a user query, a lightweight LLM produces a keyword query that is executed by a standard retriever (BM25), combining the generalization benefits of generative query rewriting with the efficiency and scalability of lexical indexing. We train the rewriting policy with reinforcement learning techniques. Across in- and out-of-domain evaluations, QueStER consistently improves over BM25 and is competitive with neural IR baselines, while maintaining strong efficiency.
2026-12-30
OpenReview (publié)
openreview.net
openreview.net
Improved mapping of Arctic fractional land cover and land cover change from multi-resolution optical remote sensing
Xiaoran Zhu
Jonathan A. Wang
Oliver Sonnentag
Isla H. Myers-Smith
Daryl Yang
Kathleen M. Orndahl
Leon Nill
Antoine Caron-Guay
Étienne Laliberté
Mark A. Friedl
Changes in Arctic tundra vegetation, driven by climate change, may be inducing major shifts in ecosystem services and the Arctic carbon budg… (voir plus)et, and altering high latitude feedbacks to the climate system. Field-based studies have documented warming-induced shrub expansion, and remote sensing has revealed heterogeneous, but primarily positive, trends in peak summer greenness across the Arctic. However, efforts to move beyond remotely sensed measures of spectral greening to quantify the spatial extent and rate of shrub expansion have been constrained by spectral similarities among tundra vegetation types, limited ground truth data, low revisit frequency of satellite observations, and sub-pixel heterogeneity of land cover at medium spatial resolution (30 m). To address these challenges, we developed a methodology that integrates high spatial resolution (2 m) commercial satellite imagery with Harmonized Landsat and Sentinel-2 observations in a machine learning framework, and used it to produce annual maps for 2016 to 2023 of sub-pixel land cover fractions at 30-m spatial resolution across three Arctic tundra ecoregions spanning 3.35 × 105 km2 between the Seward and Tuktoyaktuk Peninsulas. Uncertainty was quantified at each pixel via Monte Carlo resampling. Independent accuracy assessments yielded good accuracies (mean squared errors of 15.98% and 11.89% for low-stature vegetation and erect shrub cover, respectively), that were comparable to or exceeded previous mapping efforts. Further, repeat commercial satellite image pairs enabled the first assessment of mapped fractional cover change in Arctic tundra (R2 of 0.46 and 0.55, change direction accuracies of 77% and 78% for low-stature vegetation and erect shrub cover, respectively). This novel, scalable, multi-sensor approach to fractional land cover mapping produced the first annual maps of land cover fractions in the Arctic tundra, which support more accurate representation of vegetation dynamics and their linkages to climate change and disturbance processes.
2026-09-30
Remote Sensing of Environment (publié)
doi.org
SHINIER: An open-source Python package for controlling low-level image properties
Mathias Salvas-Hébert
Nicolas Dupuis-Roy
Catherine Landry
Ian Charest
Frédéric Gosselin
2026-08-31
SoftwareX (publié)
doi.org
CoPeP: Benchmarking Continual Pretraining for Protein Language Models
Darshan Patil
Pranshu Malviya
Mathieu Reymond
Quentin Fournier
Sarath Chandar
Protein language models (pLMs) have recently gained significant attention for their ability to uncover relationships between sequence, struc… (voir plus)ture, and function from evolutionary statistics, thereby accelerating therapeutic drug discovery. These models learn from large protein databases that are continuously updated by the biology community and whose dynamic nature motivates the application of continual learning, not only to keep up with the ever-growing data, but also as an opportunity to take advantage of the temporal meta-information that is created during this process. As a result, we introduce the Continual Pretraining of Protein Language Models (CoPeP) benchmark, a novel benchmark for evaluating continual learning approaches on pLMs. Specifically, we curate a sequence of protein datasets derived from the UniProt Knowledgebase spanning a decade and define metrics to assess pLM performance across 31 protein understanding tasks. We evaluate several methods from the continual learning literature, including replay, unlearning, and plasticity-based methods, some of which have never been applied to models and data of this scale. Our findings reveal that incorporating temporal meta-information improves perplexity by up to 7% even when compared to training on data from all tasks jointly. Moreover, even at scale, several continual learning methods outperform naive continual pretraining. The CoPeP benchmark offers an exciting opportunity to study these methods at scale in an impactful real-world application.
2026-08-24
Work-In-Progress Track @ Conference on Lifelong Learning Agents (poster)
doi.org
openreview.net
Generalization Measures under Controlled Covariate Shift: A Regime-Aware Benchmark
Sora Nakai
Youssef Fadhloun
Kacem Mathlouthi
Kotaro Yoshida
Ganesh Talluri
Ioannis Mitliagkas
Hiroki Naganuma
Predicting generalization from quantities available before target-test evaluation remains a central challenge in deep learning. The systemat… (voir plus)ic benchmark of Jiang et al. (2020) evaluated many generalization measures, but it focused on independent and identically distributed (IID) settings. We revisit this problem for image classifiers evaluated under controlled corruptions and perturbations. Our study uses CIFAR-10-C/P, where the label space and task remain fixed while the input images are degraded or perturbed. This setting also allows us to revisit the robustness concerns raised by Dziugaite et al. (2020), who showed that the apparent reliability of generalization measures can depend strongly on experimental conditions. Our experiments show that the usefulness of generalization measures is strongly regime-dependent. In our exploratory decision analysis across three CNN-style architectures, sharpness- and input-gradient-based measures are among the leading individual signals, whereas family results are close and architecture dependent. Optimization-based measures, Information Criteria, and Sharpness-based measures provide additional regime-dependent signals in correlation or local-reliability analyses. Together, these findings suggest that model selection should not rely only on measures favored by IID evaluation. Instead, within the evaluated CIFAR-10-C/P setting and architectures, generalization measures should be treated as regime-dependent ranking signals whose utility must be evaluated for the intended corruption or perturbation setting.
2026-08-24
Transactions on Machine Learning Research (accepté)
doi.org
openreview.net
Loss Smoothing for Stable Adaptation Under Distribution Shift
Darshan Patil
Ekaterina Lobacheva
Razvan Pascanu
Sarath Chandar
In settings such as fine-tuning and reinforcement learning, neural networks are often adapted under distribution shift. Standard adaptation … (voir plus)methods typically optimize the target objective directly, inducing an abrupt change from the source training objective. This abrupt transition can distort learned representations, including features that may still be useful for the new task. We investigate whether a more gradual transition can improve adaptation. We propose loss smoothing, a simple approach that interpolates between the source and target training objectives at the start of adaptation. This smooth transition helps to preserve useful features from the source distribution while still enabling the model to specialize to the target distribution. Across controlled supervised shifts, pretrained vision adaptation, offline-to-online and online reinforcement learning, and language model fine-tuning, we find that loss smoothing consistently improves performance, suggesting that smoother objective transitions are a broadly useful tool for model adaptation.
2026-08-24
Work-In-Progress Track @ Conference on Lifelong Learning Agents (poster)
doi.org
openreview.net
Evaluation Awareness in Language Models: Representation, Verbalization, and Control
Farzaneh Heidari
Amin Memarian
Guillaume Rabusseau
Both capability and safety benchmarks rest upon the assumption that the behavior of language models undergoing a test is informative about t… (voir plus)heir behavior in deployment. This assumption can fail, should models infer that they are being evaluated and condition their response on such context. This hypothesis, termed ``evaluation awareness'', has been observed in frontier and open-weight language models alike. We provide a systematic study of this phenomenon, by probing for it across six language models (from four families and three sizes) and three metrics. More precisely, we examine whether (i) being under evaluation is linearly represented within the models'activations space, (ii) it is verbalized in their output tokens (as scored by an LLM-as-judge), and (iii) steering causally affects their behavior. For the open-checkpoint Olmo models, we further test these measures at every training stage. In doing so, we report that evaluation awareness is linearly decodable from the residual streams of every model (best AUROC
2026-08-21
arXiv (prépublication)
doi.org
arxiv.org
BiDRA: Bayesian Inference as a Robust Alternative to Non-Linear Regres-sion for Dose-Response Efficiency Metrics Assessment
Caroline Labelle
Petr Smirnov
Maud David
Mario Callejo
Benjamin Haibe-Kains
Anne Marinier
Sébastien Lemieux
Abstract Motivation Dose-response metrics such as the half-maximal inhibitory concentration, high-dose response, and slope are central to dr… (voir plus)ug discovery, yet standard Levenberg-Marquardt fits often produce biased or unsupported values for incomplete curves and lack uncertainty measures. In practice, this forces experimenters to visually inspect each curve fit to judge its reliability, a tedious, subjective, and non-scalable process. Results Across 421,405 public dose-response experiments from three large pharmacogenomic datasets, shared-concentration viability responses were highly consistent across biological replicates, making it reasonable to expect derived efficiency metrics to show comparable replicate behavior when supported by the data. However, Levenberg-Marquardt fits to incomplete curves often produced unsupported metric values, including inflated rates of observable potency estimates (>70% versus ∼39% complete curves), which could falsely suggest metric-level disagreement between similar replicate responses. BiDRA represents efficacy, potency, and slope as posterior distributions, allowing uncertainty to remain large when the data do not support precise metric inference. This avoids treating uncertainty-dominated responses as conflicting point estimates, exposes unsupported Levenberg-Marquardt estimates, and supports uncertainty-aware compound ranking. We illustrate its utility in a structure–activity relationship screen, demonstrating how posteriors enable robust, criteria-based selection in drug discovery settings. Availability and implementation BiDRA is implemented in Julia and available at https://github.com/lemieux-lab/bidra_robustness.
2026-08-20
Bioinformatics Advances (publié)
doi.org
Social interactions between people of same and different generations shape longitudinal changes in interpersonal neural synchrony, loneliness, and social connection
Ryssa Moffat
Guillaume Dumas
Emily S. Cross
Loneliness is globally acknowledged as a severe and burgeoning health risk, fueling interest in helping people of all ages form meaningful s… (voir plus)ocial connections. One promising approach consists of intergenerational social programs. While behavioral and qualitative evidence derived from such programs promise health and wellbeing benefits, the physiological consequences of repeated intergenerational encounters remain unknown. Insight into physiological changes will shed light on the mechanisms of social connection. We charted longitudinal changes in interpersonal neural synchrony (INS) in 31 intergenerational (older/younger adult) and 30 same-generation (younger adult) dyads across a six-session creative drawing program. At each session, dyads completed self-report measures, drew together and alone, and had their cortical activation recorded with fNIRS. In both groups, INS was greater while dyads drew together than alone. Across sessions, intergenerational dyads' INS decreased and same-generation dyads' INS increased. INS in RIFG~RTPJ and RIFG~RIFG were predictive of loneliness levels and feelings of social closeness, respectively. This exploratory longitudinal research reinforces the multi-faceted nature of INS dynamics as social connections are forged.
2026-08-19
PLOS Biology (publié)
doi.org
SingularClip: Preventing Spectral Collapse to Maintain Plasticity in Continual and Reinforcement Learning
Tyler Kastner
Nimrod De La Vega
Amir-massoud Farahmand
Neural networks trained on nonstationary tasks frequently lose the ability to fit new targets, a phenomenon referred to as loss of plasticit… (voir plus)y. We identify a novel source of plasticity loss due to the growing anisotropy of weight matrices'singular values during training, and analyze this phenomenon both empirically and theoretically. To mitigate this issue, we introduce SingularClip, a procedure that periodically clips the singular values of all weight matrices. We show that SingularClip performs strongly against baselines across a range of tasks in both continual supervised learning and deep reinforcement learning.
2026-08-17
arXiv (prépublication)
doi.org
arxiv.org
Tuning in together: LSD enhances inter-brain synchrony and felt connectedness in romantic couples
Natasha L. Mason
Artur Czeszumski
Iva Totomanova
Filip Trbusek
Mauro Cavarra
Steph Ashton
Stefan Toennes
Eef Theunissen
Johannes Reckweg
P. L. Lockwood
Marieke DeWitte
Katrin Preller
Kim Kuypers
Guillaume Dumas
Pablo Mallaroni
Jan Ramaekers
Abstract Social connection is fundamental to human wellbeing. Serotonergic psychedelics such as lysergic acid diethylamide (LSD) acutely hei… (voir plus)ghten subjective connectedness, yet their effects on real-time social connection remain poorly understood. Using EEG hyperscanning in a randomized, double-blind, placebo-controlled crossover study, we recorded neural activity simultaneously from both members of healthy romantic couples (N=25) who received LSD (50 μg) or placebo together, across resting and interactive states. LSD increased subjective connectedness, including feelings of love, closeness, trust, and being “in sync,” while reducing loneliness, compared to placebo. This affiliative shift dissociated from the drug’s pharmacokinetic time-course, remaining elevated as subjective intensity and plasma concentration declined. In parallel, LSD increased inter-brain synchrony during shared rest, carried specifically by theta-band amplitude-envelope coupling. Importantly, this effect survived two complementary controls. First, it exceeded coupling between unrelated individuals and second the effects depended on contemporaneous neural alignment rather than shared drug-induced dynamics. Exploratory analyses showed that romantic partners with greater resting synchrony reported greater feelings of connectedness. These findings provide the first evidence that a psychedelic enhances brain-to-brain coupling between people, linking a pharmacologically induced state of felt connection to a measurable signature shared across interacting brains.
2026-08-17
bioRxiv (prépublication)
doi.org