Portrait of Simon Lacoste-Julien

Simon Lacoste-Julien

Core Academic Member

slacoste@mila.quebec

Canada CIFAR AI Chair

Associate Scientific Director, Mila, Associate Professor, Université de Montréal, Department of Computer Science and Operations Research

Vice President and Lab Director, Samsung Advanced Institute of Technology (SAIT) AI Lab, Montréal

Research Topics

Causality

Computer Vision

Deep Learning

Generative Models

Machine Learning Theory

Natural Language Processing

Optimization

Probabilistic Models

Biography

Simon Lacoste-Julien is an associate professor at Mila – Quebec Artificial Intelligence Institute and in the Department of Computer Science and Operations Research (DIRO) at Université de Montréal. He is also a Canada CIFAR AI Chair and heads (part time) the SAIT AI Lab Montréal.

Lacoste-Julien‘s research interests are machine learning and applied mathematics, along with their applications to computer vision and natural language processing. He completed a BSc in mathematics, physics and computer science at McGill University, a PhD in computer science at UC Berkeley and a postdoc at the University of Cambridge.

After spending several years as a researcher at INRIA and the École normale supérieure in Paris, he returned to his home city of Montréal in 2016 to answer Yoshua Bengio’s call to help grow the Montréal AI ecosystem.

Current Students

Reza Babanezhad Harikandeh

Independent visiting researcher - Samsung SAIT

Aristide Baratin

Independent visiting researcher - Samsung SAIT

aristidebaratin@hotmail.com

Vitoria Barin Pacela

PhD - Université de Montréal

Quentin Bertrand

Collaborating Alumni - Université de Montréal

Principal supervisor :

Independent visiting researcher - Samsung SAIT

Marwa El Halabi

Independent visiting researcher - Samsung SAIT

Jose Gallego Posada

Collaborating Alumni - Université de Montréal

PhD - Université de Montréal

Independent visiting researcher - Samsung SAIT

yashgoyal.yg1@gmail.com

Meraj Hashemizadeh

Collaborating researcher - Université de Montréal

Alexia Jolicoeur-Martineau

Independent visiting researcher - Samsung SAIT

Pedram Khorsandi

PhD - Université de Montréal

Independent visiting researcher - Seoul National University, Korea

Independent visiting researcher - Université de Montréal

Sébastien Lachapelle

PhD - Université de Montréal

Independent visiting researcher - Pohang University of Science and Technology in Pohang, Korea

Collaborating researcher

PhD - Université de Montréal

Rozhin Nobahari

Master's Research - Université de Montréal

PhD - Université de Montréal

PhD - Université de Montréal

Independent visiting researcher - Samsung SAIT

Motahareh Sohrabi

Collaborating researcher - Université de Montréal

PhD - Université de Montréal

Independent visiting researcher - Samsung SAIT

Blog Posts

Additive Decoders for Latent Variables Identification and Cartesian-Product Extrapolation

March 18, 2024

Additive Decoders for Latent Variables Identification and Cartesian-Product Extrapolation

by

Sébastien Lachapelle

Ioannis Mitliagkas

Simon Lacoste-Julien

Read the article

Publications

Negative Momentum for Improved Game Dynamics

Reyhane Askari Hemmat

Mohammad Pezeshki

Gabriel Huang

Rémi LE PRIOL

Simon Lacoste-Julien

Ioannis Mitliagkas

Games generalize the single-objective optimization paradigm by introducing different objective functions for different players. Differentiab… (see more)le games often proceed by simultaneous or alternating gradient updates. In machine learning, games are gaining new importance through formulations like generative adversarial networks (GANs) and actor-critic systems. However, compared to single-objective optimization, game dynamics are more complex and less understood. In this paper, we analyze gradient-based methods with momentum on simple games. We prove that alternating updates are more stable than simultaneous updates. Next, we show both theoretically and empirically that alternating gradient updates with a negative momentum term achieves convergence in a difficult toy adversarial problem, but also on the notoriously difficult to train saturating GANs.

2018-07-12

ArXiv (preprint)

Frank-Wolfe Splitting via Augmented Lagrangian Method

Fabian Pedregosa

Simon Lacoste-Julien

Minimizing a function over an intersection of convex sets is an important task in optimization that is often much more challenging than mini… (see more)mizing it over each individual constraint set. While traditional methods such as Frank-Wolfe (FW) or proximal gradient descent assume access to a linear or quadratic oracle on the intersection, splitting techniques take advantage of the structure of each sets, and only require access to the oracle on the individual constraints. In this work, we develop and analyze the Frank-Wolfe Augmented Lagrangian (FW-AL) algorithm, a method for minimizing a smooth function over convex compact sets related by a "linear consistency" constraint that only requires access to a linear minimization oracle over the individual constraints. It is based on the Augmented Lagrangian Method (ALM), also known as Method of Multipliers, but unlike most existing splitting methods, it only requires access to linear (instead of quadratic) minimization oracles. We use recent advances in the analysis of Frank-Wolfe and the alternating direction method of multipliers algorithms to prove a sublinear convergence rate for FW-AL over general convex compact sets and a linear convergence rate for polytopes.

2018-03-31

ArXiv (preprint)

Frank-Wolfe Splitting via Augmented Lagrangian Method

Fabian Pedregosa

Simon Lacoste-Julien

Minimizing a function over an intersection of convex sets is an important task in optimization that is often much more challenging than mini… (see more)mizing it over each individual constraint set. While traditional methods such as Frank-Wolfe (FW) or proximal gradient descent assume access to a linear or quadratic oracle on the intersection, splitting techniques take advantage of the structure of each sets, and only require access to the oracle on the individual constraints. In this work, we develop and analyze the Frank-Wolfe Augmented Lagrangian (FW-AL) algorithm, a method for minimizing a smooth function over convex compact sets related by a "linear consistency" constraint that only requires access to a linear minimization oracle over the individual constraints. It is based on the Augmented Lagrangian Method (ALM), also known as Method of Multipliers, but unlike most existing splitting methods, it only requires access to linear (instead of quadratic) minimization oracles. We use recent advances in the analysis of Frank-Wolfe and the alternating direction method of multipliers algorithms to prove a sublinear convergence rate for FW-AL over general convex compact sets and a linear convergence rate for polytopes.

2018-01-01

AISTATS (published)

proceedings.mlr.press