Publications

Robin Cohen

Benjamin Fung

Sébastien Gambs

2025-08-03

Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2 (published)

Revisiting Replay and Gradient Alignment for Continual Pre-Training of Large Language Models

Istabrak Abbes

Gopeshh Subbaraj

Matthew D Riemer

Nizar Islah

Benjamin Therien

Tsuguchika Tabaru

Hiroaki Kingetsu

Sarath Chandar

Irina Rish

Training large language models (LLMs) typically involves pre-training on massive corpora, only to restart the process entirely when new data… (see more) becomes available. A more efficient and resource-conserving approach would be continual pre-training, where models are updated with new data rather than retraining from scratch. However, the introduction of new data often causes distribution shifts, leading to performance degradation on previously learned tasks. In this paper, we take a deeper look at two popular proposals for addressing this distribution shift within the continual learning literature: experience replay and gradient alignment. We consider continual pre-training of models within the Llama family of architectures at a large scale across languages with 100 billion tokens of training data in each language, finding that both replay and gradient alignment lead to more stable learning without forgetting. This conclusion holds both as we vary the model scale and as we vary the number and diversity of tasks. Moreover, we are the first to demonstrate the effectiveness of gradient alignment techniques in the context of LLM pre-training and propose an efficient implementation of meta-experience replay (MER) that imbues experience replay with the benefits of gradient alignment despite negligible compute and memory overhead. Our scaling analysis across model sizes and replay rates indicates that small rates of replaying old examples are definitely a more valuable use of compute than investing in model size, but that it is more compute efficient to scale the size of the model than invest in high rates of replaying old examples.

2025-08-03

ArXiv (preprint)

Revisiting Replay and Gradient Alignment for Continual Pre-Training of Large Language Models

Istabrak Abbes

Gopeshh Subbaraj

Matthew D Riemer

Nizar Islah

Benjamin Therien

Tsuguchika Tabaru

Hiroaki Kingetsu

Sarath Chandar

Irina Rish

2025-08-03

ArXiv (preprint)

Temporal Graph Learning Workshop

Shenyang Huang

Daniele Zambon

Andrea Cini

Farimah Poursafaei

Jacob Chmura

Julia Gastinger

Micheal Bronstein

2025-08-03

Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2 (published)

Temporal Graph Learning Workshop

Shenyang Huang

Daniele Zambon

Andrea Cini

Farimah Poursafaei

Jacob Chmura

Julia Gastinger

Micheal Bronstein

2025-08-03

Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2 (published)

Temporal Graph Learning Workshop

Shenyang Huang

Daniele Zambon

Andrea Cini

Farimah Poursafaei

Jacob Chmura

Julia Gastinger

Michael Bronstein

2025-08-03

Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2 (published)

From Technical Excellence to Practical Adoption: Lessons Learned Building an ML-Enhanced Trace Analysis Tool

Kaveh Shahedi

Matthew Khouzam

Heng Li

Maxime Lamothe

Foutse Khomh

2025-08-02

ArXiv (preprint)

From Technical Excellence to Practical Adoption: Lessons Learned Building an ML-Enhanced Trace Analysis Tool

Kaveh Shahedi

Matthew Khouzam

Heng Li

Maxime Lamothe

Foutse Khomh

System tracing has become essential for understanding complex software behavior in modern systems, yet sophisticated trace analysis tools fa… (see more)ce significant adoption gaps in industrial settings. Through a year-long collaboration with Ericsson Montr\'eal, developing TMLL (Trace-Server Machine Learning Library, now in the Eclipse Foundation), we investigated barriers to trace analysis adoption. Contrary to assumptions about complexity or automation needs, practitioners struggled with translating expert knowledge into actionable insights, integrating analysis into their workflows, and trusting automated results they could not validate. We identified what we called the Excellence Paradox: technical excellence can actively impede adoption when conflicting with usability, transparency, and practitioner trust. TMLL addresses this through adoption-focused design that embeds expert knowledge in interfaces, provides transparent explanations, and enables incremental adoption. Validation through Ericsson's experts'feedback, Eclipse Foundation's integration, and a survey of 40 industry and academic professionals revealed consistent patterns: survey results showed that 77.5% prioritize quality and trust in results over technical sophistication, while 67.5% prefer semi-automated analysis with user control, findings supported by qualitative feedback from industrial collaboration and external peer review. Results validate three core principles: cognitive compatibility, embedded expertise, and transparency-based trust. This challenges conventional capability-focused tool development, demonstrating that sustainable adoption requires reorientation toward adoption-focused design with actionable implications for automated software engineering tools.

2025-08-02

ArXiv (preprint)

Inhibition of epithelial cell YAP-TEAD/LOX signaling attenuates pulmonary fibrosis in preclinical models

Darcy Elizabeth Wagner

Hani N. Alsafadi

Nilay Mitash

Aurelien Justet

Qianjiang Hu

Ricardo Pineda

Claudia Staab-Weijnitz

Martina Korfei

Nika Gvazava

Kristin Wannemo

Ugochi Onwuka

Molly Mozurak

Adriana Estrada-Bernal

Juan Cala Garcia

Katrin Mutze

Rita Costa

Deniz Bölükbas

John Stegmayr

Wioletta Skronska-Wasek

Stephan Klee … (see 14 more)

Chiharu Ota

Hoeke A. Baarsma

Jingtao Wang

John Sembrat

Anne Hilgendorff

Jun Ding

Andreas Günther

Rachel Chambers

Ivan O Rosas

Stijn de Langhe

Naftali Kaminski

Mareike Lehmann

Oliver Eickelberg

Melanie Königshoff

Idiopathic pulmonary fibrosis (IPF) is a progressive and lethal disease characterized by excessive extracellular matrix deposition. Current … (see more)IPF therapies slow disease progression but do not stop or reverse it. The (myo)fibroblasts are thought to be the main cellular contributors to excessive extracellular matrix production in IPF. Here we show that fibrotic alveolar type II cells regulate production and crosslinking of extracellular matrix via the co-transcriptional activator YAP. YAP leads to increased expression of Lysl oxidase (LOX) and subsequent LOX-mediated crosslinking by fibrotic alveolar type II cells. Pharmacological YAP inhibition via verteporfin reverses fibrotic alveolar type II cell reprogramming and LOX expression in experimental lung fibrosis in vivo and in human fibrotic tissue ex vivo. We thus identify YAP-TEAD/LOX inhibition in alveolar type II cells as a promising potential therapy for IPF patients.

2025-08-02

Nature Communications (published)

Adaptation, Comparison and Practical Implementation of Fairness Schemes in Kidney Exchange Programs

William St-Arnaud

Margarida Carvalho

Golnoosh Farnadi

In Kidney Exchange Programs (KEPs), each participating patient is registered together with an incompatible donor. Donors without an incompat… (see more)ible patient can also register. Then, KEPs typically maximize overall patient benefit through donor exchanges. This aggregation of benefits calls into question potential individual patient disparities in terms of access to transplantation in KEPs. Considering solely this utilitarian objective may become an issue in the case where multiple exchange plans are optimal or near-optimal. In fact, current KEP policies are all-or-nothing, meaning that only one exchange plan is determined. Each patient is either selected or not as part of that unique solution. In this work, we seek instead to find a policy that contemplates the probability of patients of being in a solution. To guide the determination of our policy, we adapt popular fairness schemes to KEPs to balance the usual approach of maximizing the utilitarian objective. Different combinations of fairness and utilitarian objectives are modelled as conic programs with an exponential number of variables. We propose a column generation approach to solve them effectively in practice. Finally, we make an extensive comparison of the different schemes in terms of the balance of utility and fairness score, and validate the scalability of our methodology for benchmark instances from the literature.

2025-08-01

European Journal of Operational Research (published)

Efficient Deep Reinforcement Learning-Based Supplementary Damping Control With a Coordinated RMS Training and EMT Testing Scheme

Tao Xue

Mingxuan Zhao

Ilhan Kocar

Mohsen Ghafouri

Antoine Lesage-Landry

Siqi Bu

Ziqing Zhu

Inverter-based resources (IBRs) can cause instability in weak AC grids. While supplementary damping controllers (SDCs) effectively mitigate … (see more)this instability, they are typically designed for specific resonance frequencies but struggle with large shifts caused by changing grid conditions. This paper proposes a deep reinforcement learning-based agent (DRL Agent) as an adaptive SDC to handle shifted resonance frequencies. To address the time-consuming nature of training DRL Agents in electromagnetic transient (EMT) simulations, we coordinate fast root mean square (RMS) and EMT simulations. Resonance frequencies of the weak grid instability are accurately reproduced by RMS simulations to support the training process. The DRL Agent’s efficacy is tested in unseen scenarios outside the training dataset. We then iteratively improve the DRL Agent’s performance by modifying the reward function and hyper-parameters.

2025-08-01

IEEE Transactions on Power Delivery (published)

Efficient Deep Reinforcement Learning-Based Supplementary Damping Control with a Coordinated RMS Training and EMT Testing Scheme

Tao Xue

Mingxuan Zhao

Ilhan Kocar

Mohsen Ghafouri

Antoine Lesage-Landry

Siqi Bu

Ziqing Zhu

2025-08-01

IEEE Transactions on Power Delivery (published)