Simon Lacoste-Julien

aristidebaratin@hotmail.com

Biographie

Simon Lacoste-Julien est professeur agrégé au Département d'informatique et de recherche opérationnelle (DIRO) de l'Université de Montréal, membre cofondateur de Mila – Institut québécois d’intelligence artificielle et titulaire d'une chaire en IA Canada-CIFAR. Il dirige également à temps partiel le SAIT AI Lab Montréal.

Ses recherches portent sur l'apprentissage automatique et les mathématiques appliquées, et intègrent des applications à la vision artificielle et au traitement du langage naturel. Il a obtenu une licence en mathématiques, physique et informatique à l’Université McGill, un doctorat en informatique à l’Université de Californie à Berkeley et un postdoctorat à l'Université de Cambridge.

Il a passé quelques années à l'Institut national de recherche en sciences et technologies du numérique (INRIA) et à l'École normale supérieure de Paris en tant que professeur de recherche avant de revenir à Montréal, en 2016, pour répondre à l'appel de Yoshua Bengio et contribuer à la croissance de l'écosystème de l'IA à Montréal.

Étudiants actuels

Reza Babanezhad Harikandeh

Visiteur de recherche indépendant - Samsung SAIT

babanezr@mila.quebec

Aristide Baratin

Visiteur de recherche indépendant - Samsung SAIT

Doctorat - UdeM

vitoria.barin-pacela@mila.quebec

Doctorat - UdeM

simon.dufort-labbe@mila.quebec

Marwa El Halabi

Visiteur de recherche indépendant - Samsung SAIT

marwa.el-halabi@mila.quebec

Doctorat - UdeM

antonio-miguel.gois@mila.quebec

Yash Goyal

Visiteur de recherche indépendant - Samsung SAIT

yashgoyal.yg1@gmail.com

Meraj Hashemizadeh

Collaborateur·rice de recherche - UdeM

merajhse@mila.quebec

fahimeh.hosseini@mila.quebec

Fahimeh HosseiniNoohdani

Collaborateur·rice de recherche - UdeM

pedram.khorsandi@mila.quebec

Pedram Khorsandi

Doctorat - UdeM

Boris Knyazev

Visiteur de recherche indépendant - UdeM

Visiteur de recherche indépendant - Samsung - SAIT

lachaseb@mila.quebec

yukti.makhija@mila.quebec

Lucas Maes

Doctorat - UdeM

lucas.maes@mila.quebec

Doctorat - UdeM

Co-superviseur⋅e :

Ioannis Mitliagkas

Doctorat - UdeM

Co-superviseur⋅e :

Aaron Courville

alan.milligan@mila.quebec

Collaborateur·rice alumni - UdeM

rozhin.nobahari@mila.quebec

Juan Ramirez

Doctorat - UdeM

juan.ramirez@mila.quebec

Doctorat - UdeM

mansi.rankawat@mila.quebec

Visiteur de recherche indépendant - Univeristy of Tübingen

Theo Saulus

Doctorat - UdeM

Co-superviseur⋅e :

Dhanya Sridhar

theo.saulus@mila.quebec

motahareh.sohrabi@mila.quebec

Damien Scieur

Visiteur de recherche indépendant - Samsung SAIT

Collaborateur·rice de recherche - UdeM

Yan Zhang

Visiteur de recherche indépendant - Samsung SAIT

yan.zhang@mila.quebec

tianyue.zhang@mila.quebec

Tianyue (Helen) Zhang

Doctorat - UdeM

Décodeurs additifs pour l’identification des variables latentes et l’extrapolation du produit cartésien

Billets de blogue

Additive Decoders for Latent Variables Identification and Cartesian-Product Extrapolation

18 mars 2024

par

Sébastien Lachapelle

Divyat Mahajan

Ioannis Mitliagkas

Simon Lacoste-Julien

Lire l'article

Publications

Accelerated and Stable Convergence with Anchored Optimistic Method

Jianxin You

We study first-order methods for solving monotone variational inequalities arising in min-max optimization. Classical approaches such as the… (voir plus) extragradient method rely on two gradient queries per iteration, which limits their analysis and applicability in the online and stochastic settings. We propose a family of Generalized Optimistic Methods with Anchoring (GOMA), which combine two-time-scale optimistic updates with an anchoring term inspired by Halpern iteration. In the deterministic setting, GOMA achieves the optimal accelerated last-iterate rate

2026-06-18

arXiv (prépublication)

Unsupervised Causal Abstractions Discovery

Theo Saulus

Dhanya Sridhar

Causal abstractions formalize when a high-level structural causal model (SCM) captures the interventional behavior of a lower-level SCM. Exi… (voir plus)sting applications of this notion largely follow a hypothesis-testing paradigm: an expert proposes a candidate high-level model and then evaluates if the low-level system implements it. We study the complementary problem of learning a high-level model directly from low-level measurements. Our contributions leverage hypotheses from low-rank causal discovery, and can be summarized as follows: (1) we show that observations generated by a low-rank graph induce latents that form a causal abstraction, (2) we provide identifiability results about these latents, and (3) we propose a practical objective to learn this high-level SCM.

2026-06-16

arXiv (prépublication)

Reparametrizing Shampoo and SOAP for Subspace Basis Updates and BFloat16 Storage

Alan Milligan

Zikun Xu

Felix Dangel

Wu Lin

Shampoo-based methods, such as KL-Shampoo and SOAP, have demonstrated strong performance in training neural networks and rely on QR decompos… (voir plus)ition. Because existing QR implementations require single-precision (FP32) arithmetic and remain computationally expensive, these methods become time- and memory-intensive when their preconditioning matrices are large. Moreover, using BFloat16 (BFP16) storage to reduce memory usage can degrade the performance of Shampoo-based methods. We propose a reparametrization of the preconditioner that supports BFP16 storage and forms a complete basis by combining updated basis vectors with unchanged ones. By updating only part of the basis through QR decomposition in a subspace, our approach reduces computational overhead while mitigating the performance degradation caused by BFP16 storage. Our approach applies broadly to Shampoo-based methods that employ QR decomposition, including KL-Shampoo, SOAP, and KL-SOAP. In particular, it improves the performance of SOAP and KL-SOAP under BFP16 storage, enabling KL-SOAP to match or exceed KL-Shampoo. Overall, our approach makes Shampoo-based methods more memory- and time-efficient.

2026-05-24

arXiv (prépublication)

From Lyapunov Analysis to Algorithm Design in two-sided PL Minimax Optimization

Mansi Rankawat

Michael Muehlebach

Damien Scieur

We derive algorithms for smooth nonconvex nonconcave minimax optimization and establish linear convergence rates for problems that satisfy t… (voir plus)he two-sided Polyak-Lojasiewicz (PL) inequality. At the core of our approach is the observation that Lyapunov functions can be used not only to certify convergence a posteriori, but also to design algorithms. By replacing an idealized, intractable Lyapunov function with a computable surrogate based on gradient information, we derive TALDA (Tri-Action Lyapunov Descent Ascent), a single-loop algorithm that enforces Lyapunov descent by construction. TALDA guarantees linear convergence under the two-sided PL condition, with a rate that depends explicitly on the cross-smoothness constant. This recovers existing worst-case guarantees while yielding sharper convergence rates in weakly coupled min–max problems.

2026-04-29

International Conference on Machine Learning (accepté)

Stop Probing, Start Coding: Why Linear Probes and Sparse Autoencoders Fail at Compositional Generalisation

Vitoria Barin Pacela

Shruti Joshi

Isabela Camacho

David Klindt

The linear representation hypothesis states that neural network activations encode high-level concepts as linear mixtures. However, under su… (voir plus)perposition, this encoding is a projection from a higher-dimensional concept space into a lower-dimensional activation space, and a linear decision boundary in the concept space need not remain linear after projection. In this setting, classical sparse coding methods with per-sample iterative inference leverage compressed sensing guarantees to recover latent factors. Sparse autoencoders (SAEs), on the other hand, amortise sparse inference into a fixed encoder, introducing a systematic gap. We show this amortisation gap persists across training set sizes, latent dimensions, and sparsity levels, causing SAEs to fail under out-of-distribution (OOD) compositional shifts. Through controlled experiments that decompose the failure, we identify dictionary learning -- not the inference procedure -- as the binding constraint: SAE-learned dictionaries point in substantially wrong directions, and replacing the encoder with per-sample FISTA on the same dictionary does not close the gap. An oracle baseline proves the problem is solvable with a good dictionary at all scales tested. Our results reframe the SAE failure as a dictionary learning challenge, not an amortisation problem, and point to scalable dictionary learning as the key open problem for sparse inference under superposition.

2026-03-29

arXiv (prépublication)

Lloyd's $K$-Means Clustering Algorithm is Frank-Wolfe in Disguise

Michael Pokojovy

J. Marcus Jobe

Lloyd's …

2026-02-02

International Conference on Artificial Intelligence and Statistics (poster)

The Role of Causal Features in Strategic Classification for Robustness and Alignment

Sophia Gunluk

Antonio Gois

Nir Rosenfeld

Nidhi Hegde

Dhanya Sridhar

In strategic classification, an institution (e.g., a bank) anticipates adaptation from users who change their features to increase utility i… (voir plus)n a classification task (e.g., loan repayment). Since a key challenge is the distribution shift induced by users, we turn to causal models, which have been shown to bound the worst-case out-of-distribution (OOD) risk, and establish several new results that link causality and strategic classification. First, we show that causal classification leads to optimal classification error after any sufficiently large adaptation, when the noise is bounded in a certain way. Second, when these assumptions do not hold, we show OOD cross-entropy risk of optimal classifiers decomposes into an OOD bias term and a term arising from not using all observable features, allowing us to determine when causal classifiers have an advantage. Finally, we show that causal classifiers can align long-term incentives between institutions and users, contrasting with previous work that highlights social costs of such approaches. We validate our theory empirically on synthetic data, finding that our results predict behavior in practice.

2026-02-02

Artificial Intelligence and Statistics (poster)

Accelerated and Stable Convergence with Anchored Generalized Optimistic Method

Jianxin You

We study first-order methods for solving monotone variational inequalities arising in min-max optimization. Classical approaches such as the… (voir plus) extragradient method rely on two gradient queries per iteration, which limits their analysis and applicability in the online and stochastic settings. We propose a family of Generalized Optimistic Methods with Anchoring (GOMA), which combine two time-scale optimistic updates with an anchoring term inspired by Halpern iteration. In particular, we show that for monotone Lipschitz operators, GOMA achieves an accelerated last-iterate convergence rate of

2025-12-31

International Conference on Machine Learning (Accept (regular))

Dual Optimistic Ascent (PI Control) is the Augmented Lagrangian Method in Disguise

Juan Ramirez

Constrained optimization is a powerful framework for enforcing requirements on neural networks. These constrained deep learning problems are… (voir plus) typically solved using first-order methods on their min-max Lagrangian formulation, but such approaches often suffer from oscillations and can fail to find all local solutions. While the Augmented Lagrangian method (ALM) addresses these issues, practitioners often favor dual optimistic ascent schemes (PI control) on the standard Lagrangian, which perform well empirically but lack formal guarantees. In this paper, we establish a previously unknown equivalence between these approaches: dual optimistic ascent on the Lagrangian is equivalent to gradient descent-ascent on the Augmented Lagrangian. This finding allows us to transfer the robust theoretical guarantees of the ALM to the dual optimistic setting, proving it converges linearly to all local solutions. Furthermore, the equivalence provides principled guidance for tuning the optimism hyper-parameter. Our work closes a critical gap between the empirical success of dual optimistic methods and their theoretical foundation.

2025-12-31

International Conference on Learning Representations (Accept (Poster))

Operationalizing Quantized Disentanglement

Vitória Barin-Pacela

Kartik Ahuja

P Vincent

2025-11-24

ArXiv (prépublication)

Tight Lower Bounds and Improved Convergence in Performative Prediction

Performative prediction is a framework accounting for the shift in the data distribution induced by the prediction of a model deployed in th… (voir plus)e real world. Ensuring rapid convergence to a stable solution where the data distribution remains the same after the model deployment is crucial, especially in evolving environments. This paper extends the Repeated Risk Minimization (RRM) framework by utilizing historical datasets from previous retraining snapshots, yielding a class of algorithms that we call Affine Risk Minimizers and enabling convergence to a performatively stable point for a broader class of problems. We introduce a new upper bound for methods that use only the final iteration of the dataset and prove for the first time the tightness of both this new bound and the previous existing bounds within the same regime. We also prove that utilizing historical datasets can surpass the lower bound for last iterate RRM, and empirically observe faster convergence to the stable point on various performative prediction benchmarks. We offer at the same time the first lower bound analysis for RRM within the class of Affine Risk Minimizers, quantifying the potential improvements in convergence speed that could be achieved with other variants in our framework.

2025-09-17

Conference on Neural Information Processing Systems (poster)

Alexia Jolicoeur-Martineau

Understanding Adam Requires Better Rotation Dependent Assumptions

Tianyue H. Zhang

Lucas Maes

Alan Milligan

Ioannis Mitliagkas

Damien Scieur

Charles Guille-escuret

Despite its widespread adoption, Adam's advantage over Stochastic Gradient Descent (SGD) lacks a comprehensive theoretical explanation. This… (voir plus) paper investigates Adam's sensitivity to rotations of the parameter space. We observe that Adam's performance in training transformers degrades under random rotations of the parameter space, indicating a crucial sensitivity to the choice of basis in practice. This reveals that conventional rotation-invariant assumptions are insufficient to capture Adam's advantages theoretically. To better understand the rotation-dependent properties that benefit Adam, we also identify structured rotations that preserve or even enhance its empirical performance. We then examine the rotation-dependent assumptions in the literature and find that they fall short in explaining Adam's behaviour across various rotation types. In contrast, we verify the orthogonality of the update as a promising indicator of Adam's basis sensitivity, suggesting it may be the key quantity for developing rotation-dependent theoretical frameworks that better explain its empirical success.

2025-09-17

Conference on Neural Information Processing Systems (poster)