The upcoming meeting, taking place on November 10 at Mila, will explore how we can collectively develop, govern, and deploy high-performing, reliable, and secure agentic systems by connecting academic researchers, industry experts, and practitioners.
Join us on the Venture Scientist Bootcamp, a full time, 4-month incubator at Mila, built specifically for deep tech founders with elite STEM backgrounds. You have until November 21, 2026 to apply!
We use cookies to analyze the browsing and usage of our website and to personalize your experience. You can disable these technologies at any time, but this may limit certain functionalities of the site. Read our Privacy Policy for more information.
Setting cookies
You can enable and disable the types of cookies you wish to accept. However certain choices you make could affect the services offered on our sites (e.g. suggestions, personalised ads, etc.).
Essential cookies
These cookies are necessary for the operation of the site and cannot be deactivated. (Still active)
Analytics cookies
Do you accept the use of cookies to measure the audience of our sites?
Multimedia Player
Do you accept the use of cookies to display and allow you to watch the video content hosted by our partners (YouTube, etc.)?
Publications
Efficient Bilevel Optimization with KFAC-Based Hypergradients
Bilevel optimization (BO) is widely applicable to many machine learning problems. Scaling BO, however, requires repeatedly computing hypergr… (see more)adients, which involves solving inverse Hessian-vector products (IHVPs). In practice, these operations are often approximated using crude surrogates such as one-step gradient unrolling or identity/short Neumann expansions, which discard curvature information. We build on implicit function theorem-based algorithms and propose to incorporate Kronecker-factored approximate curvature (KFAC), yielding curvature-aware hypergradients with a better performance efficiency trade-off than Conjugate Gradient (CG) or Neumann methods and consistently outperforming unrolling. We evaluate this approach across diverse tasks, including meta-learning and AI safety problems. On models up to BERT, we show that curvature information is valuable at scale, and KFAC can provide it with only modest memory and runtime overhead. Our implementation is available at https://github.com/liaodisen/NeuralBo.
2026-08-29
International Conference on Artificial Intelligence and Statistics (published)
Learning Physical Operators using Neural Operators
Vignesh Gopakumar
Ander Gray
Daniel Giles
Lorenzo Zanisi
Matt J. Kusner
Timo Betcke
Stanislas Pamela
Marc Peter Deisenroth
Neural operators have emerged as promising surrogate models for solving partial differential equations (PDEs), but struggle to generalise be… (see more)yond training distributions and are often constrained to a fixed temporal discretisation. This work introduces a physics-informed training framework that addresses these limitations by decomposing PDEs using operator splitting methods, training separate neural operators to learn individual non-linear physical operators while approximating linear operators with fixed finite-difference convolutions. This modular mixture-of-experts architecture enables generalisation to novel physical regimes by explicitly encoding the underlying operator structure. We formulate the modelling task as a neural ordinary differential equation (ODE) where these learned operators constitute the right-hand side, enabling continuous-in-time predictions through standard ODE solvers and implicitly enforcing PDE constraints. Demonstrated on incompressible and compressible Navier--Stokes equations, our approach achieves better convergence and superior performance when generalising to unseen physics. The method remains parameter-efficient, enabling temporal extrapolation beyond training horizons, and provides interpretable components whose behaviour can be verified against known physics.
2026-08-29
International Conference on Artificial Intelligence and Statistics (published)
Pediatric serious illness communication (SIC) is critically important, yet scalable communication training for clinicians remains limited. C… (see more)ompared with other dialogue simulation settings, pediatric SIC poses additional challenges, including multi-party interactions, response to parental distress and strong dependence on feedback dynamics. Existing LLM-based simulators optimize generic dialogue quality rather than curriculum-contingent behavior required for effective SIC training. In collaboration with educators and pediatric clinicians, we introduce the first benchmark suite and simulation framework tailored to pediatric SIC training. Our benchmarks, PitfallBench and DialogueBench, evaluate simulators both at the turn-level and across full dialogues. We further propose SIC-Agents, a self-improving framework that generates a clinician-editable skill document to guide simulator behavior. Our experiments show that SIC-Agents outperforms static expert prompting. To support future research, we release our benchmarks for parent simulation in pediatric SIC at https://github.com/Beikewzh/sic-benchmarks
BACKGROUND
Waldenström macroglobulinemia (WM) is a rare lymphoplasmacytic lymphoma, that can lead to life-threatening complications requiri… (see more)ng intensive care unit (ICU) admission. This study aimed to describe the clinical characteristics, WM-specific complications, and outcomes of WM patients admitted to the ICU.
METHODS
We conducted a multicenter retrospective study in 6 French ICUs including all patients with WM admitted between 2010 and 2024. The primary outcome was ICU mortality, analyzed using Ridge logistic regression. One-year mortality and associated risk factors were also assessed after ICU admission.
RESULTS
A total of 151 patients with WM were admitted to the ICU over the 14-year period. ICU mortality was 20% (n = 30), with infections being the leading cause of death. The main reason for admission was acute respiratory failure, and nearly half of the patients presented with sepsis at admission. The most WM-specific complications included hyperviscosity syndrome (n = 29, 19%) and other IgM-related vascular complications (n = 27, 18%). In Ridge logistic regression, a higher SOFA score (β = 0.099) and autoimmune haemolytic anemia (β = 0.058) were most strongly associated with ICU mortality, whereas, hyperviscosity syndrome (β = -0.045) showed the strongest negative association. The one-year mortality rate was 40% (n = 58) and remission at one year was achieved in 47% (n = 56) of treated patients.
CONCLUSION
This is the first large retrospective cohort of critically ill patients with WM. SOFA score and autoimmune cytopenias were most strongly associated with ICU mortality, highlighting the importance of both disease-specific complications and baseline severity, and supporting the need for tailored prognostic tools and individualized management strategies.
Recent advances in large language models are enabling autonomous clinical agents to perform increasingly complex electronic health record (E… (see more)HR) modeling workflows. However, agents deployed at individual hospitals remain constrained by institution-specific data and modeling environments, while direct cross-hospital collaboration is restricted by the sensitivity of patient-level EHR data. Although federated learning (FL) provides a natural foundation for privacy-preserving collaboration, existing approaches remain predominantly model-centric, limiting federation to prediction models or their updates while overlooking the richer modeling experience accumulated by autonomous agents. To address this limitation, we propose FedEHR-Agents, an experience-centric federated agentic optimization framework for automated EHR modeling. Each hospital deploys an autonomous clinical EHR agent that performs data preprocessing and model development while refining local clinical modeling experience through historical memory, task-specific evaluation, and TextGrad-based prompt refinement. The federated server performs evidence-guided experience aggregation to integrate reliable and complementary modeling experience across heterogeneous hospitals and distills the aggregated experience into global meta-prompts for subsequent local refinement. Extensive experiments on real-world multi-hospital EHR benchmarks demonstrate that FedEHR-Agents consistently outperforms local and federated baselines across diverse clinical prediction tasks and remains robust across different federation scales and LLM backbones. These results establish clinical modeling experience as a promising collaborative object beyond conventional parameter-centric FL and point toward federated autonomous clinical intelligence.
Abstract Human reasoning often involves working over limited information to arrive at probabilistic conclusions. In its simplest form, this … (see more)involves making an inference that is not strictly entailed by a premise, but rather only likely given the premise. While reasoning LLMs have demonstrated strong performance on logical and mathematical tasks, their behavior on such open-ended, non-deterministic inferences remains largely unexplored. We introduce PROBCOPA, a dataset of 210 handcrafted probabilistic inferences in English, each annotated for inference likelihood by 25–30 human participants. We find that human responses are graded and varied, revealing probabilistic judgments of the inferences in our dataset. Comparing these judgments with responses from eight state-of-the-art reasoning LLMs, we show that models consistently fail to produce human-like distributions. Finally, analyzing LLM reasoning chains, we find evidence of a common reasoning pattern used to evaluate such inferences. Our findings reveal persistent differences between humans and LLMs, and underscore the need to evaluate reasoning beyond deterministic settings.1
2026-08-27
Transactions of the Association for Computational Linguistics (published)
Despite extensive safety alignment, large language models (LLMs) remain vulnerable to jailbreak attacks that bypass safeguards to elicit har… (see more)mful content. While prior work attributes this vulnerability to safety training limitations, the internal mechanisms by which LLMs process adversarial prompts remain poorly understood. We present a mechanistic analysis of the jailbreaking behavior in a large-scale, safety-aligned LLM, focusing on LLaMA-2-7B-chat-hf. Leveraging edge attribution patching and subnetwork probing, we systematically identify computational circuits responsible for generating affirmative responses to jailbreak prompts. Ablating these circuits during the first token prediction can reduce attack success rates by up to 80\%, demonstrating its critical role in safety bypass. Our analysis uncovers key attention heads and MLP pathways that mediate adversarial prompt exploitation, revealing how important tokens propagate through these components to override safety constraints. These findings advance the understanding of adversarial vulnerabilities in aligned LLMs and pave the way for targeted, interpretable defense mechanisms based on mechanistic interpretability.
In humans, chronic pain is associated with multisensory hypersensitivity, which can be modulated by sensorial exposure. Although cats experi… (see more)ence chronic pain, their sensorial perception under this condition has never been investigated. Adult neutered healthy ( n = 6) or osteoarthritic ( n = 32) cats were assessed for functional alteration, central sensitization, and sensorial (olfactory and auditory) tolerance. The effects of specific light wavelengths were tested for neuromodulation via visual exposure. Hypersensitized cats displayed stronger unpleasant odor avoidance and lower sensitivity to repeated repulsive sounds. Odor avoidance correlated with delta and theta band proportions, inverse markers of endogenous inhibitory control. Functional alteration correlated with beta activity, a pain marker. After green light exposure, neuro-sensitization decreased for 80% of the less sensitized cats, and delta and theta band proportions increased further (no longer requiring inhibition), while beta and gamma band proportions decreased. Neuro-sensitization appears central in feline osteoarthritis pain, is associated with sensorial modifications, and is modulable by non-pharmacological approaches.
Artificial neural networks (ANNs) are an important tool for studying neural computation, but many features of the brain are not captured by … (see more)standard ANN architectures. One notable missing feature in most ANN models is top-down feedback, that is projections from higher-order layers to lower-order layers in the network. Top-down feedback is ubiquitous in the brain, and it has a unique modulatory impact on activity in neocortical pyramidal neurons. However, we still do not understand its computational role. Here, we develop a deep neural network model that captures the core functional properties of top-down feedback in the neocortex, allowing us to construct hierarchical recurrent ANN models that more closely reflect the architecture of the brain. We use this to explore the impact of different hierarchical recurrent architectures on an audiovisual integration task. We find that certain hierarchies, namely those that mimic the architecture of the human brain, impart ANN models with a light visual bias similar to that seen in humans. This bias does not impair performance on the audiovisual tasks. The results further suggest that different configurations of top-down feedback make otherwise identically connected models functionally distinct from each other, and from traditional feedforward and laterally recurrent models. Altogether, our findings demonstrate that modulatory top-down feedback is a computationally relevant feature of biological brains, and that incorporating it into ANNs affects their behavior and constrains the solutions it is likely to discover.
Few-Shot Class-Incremental Learning (FSCIL) requires models to learn new classes from limited samples while retaining prior knowledge under … (see more)strict compute and memory constraints. Existing approaches face a difficult trade-off: fine-tuning is efficient but forgets, replay-based methods mitigate forgetting at high cost, and exemplar-free methods sacrifice adaptability by freezing most of the backbone. We propose Selective Backpropagation (SBP), a deterministic parameter budgeting framework that restricts gradient updates to a pre-allocated subset of parameters, freezing past knowledge and preserving unbiased capacity for future learning without costly mask optimization. SBP achieves state-of-the-art accuracy across standard FSCIL benchmarks while requiring training time close to naive fine-tuning. Crucially, we show that methods relying on frozen backbones fail in cross-domain FSCIL, where distribution shift is unavoidable, while SBP remains highly adaptive and significantly outperforms all competing approaches.
2026-08-24
Work-In-Progress Track @ Conference on Lifelong Learning Agents (poster)
Protein language models (pLMs) have recently gained significant attention for their ability to uncover relationships between sequence, struc… (see more)ture, and function from evolutionary statistics, thereby accelerating therapeutic drug discovery. These models learn from large protein databases that are continuously updated by the biology community and whose dynamic nature motivates the application of continual learning, not only to keep up with the ever-growing data, but also as an opportunity to take advantage of the temporal meta-information that is created during this process. As a result, we introduce the Continual Pretraining of Protein Language Models (CoPeP) benchmark, a novel benchmark for evaluating continual learning approaches on pLMs. Specifically, we curate a sequence of protein datasets derived from the UniProt Knowledgebase spanning a decade and define metrics to assess pLM performance across 31 protein understanding tasks. We evaluate several methods from the continual learning literature, including replay, unlearning, and plasticity-based methods, some of which have never been applied to models and data of this scale. Our findings reveal that incorporating temporal meta-information improves perplexity by up to 7% even when compared to training on data from all tasks jointly. Moreover, even at scale, several continual learning methods outperform naive continual pretraining. The CoPeP benchmark offers an exciting opportunity to study these methods at scale in an impactful real-world application.
2026-08-24
Work-In-Progress Track @ Conference on Lifelong Learning Agents (poster)