Portrait of Simon Lacoste-Julien

Simon Lacoste-Julien

Core Academic Member

slacoste@mila.quebec

Canada CIFAR AI Chair

Associate Scientific Director, Mila, Associate Professor, Université de Montréal, Department of Computer Science and Operations Research

Vice President and Lab Director, Samsung Advanced Institute of Technology (SAIT) AI Lab, Montréal

Research Topics

Causality

Computer Vision

Deep Learning

Generative Models

Machine Learning Theory

Natural Language Processing

Optimization

Probabilistic Models

Biography

Simon Lacoste-Julien is an associate professor at Mila – Quebec Artificial Intelligence Institute and in the Department of Computer Science and Operations Research (DIRO) at Université de Montréal. He is also a Canada CIFAR AI Chair and heads (part time) the SAIT AI Lab Montréal.

Lacoste-Julien‘s research interests are machine learning and applied mathematics, along with their applications to computer vision and natural language processing. He completed a BSc in mathematics, physics and computer science at McGill University, a PhD in computer science at UC Berkeley and a postdoc at the University of Cambridge.

After spending several years as a researcher at INRIA and the École normale supérieure in Paris, he returned to his home city of Montréal in 2016 to answer Yoshua Bengio’s call to help grow the Montréal AI ecosystem.

Current Students

Reza Babanezhad Harikandeh

Independent visiting researcher - Samsung SAIT

Aristide Baratin

Independent visiting researcher - Samsung SAIT

aristidebaratin@hotmail.com

Vitoria Barin Pacela

PhD - Université de Montréal

Quentin Bertrand

Collaborating Alumni - Université de Montréal

Principal supervisor :

Independent visiting researcher - Samsung

Marwa El Halabi

Independent visiting researcher - Samsung SAIT

Jose Gallego Posada

Collaborating Alumni - Université de Montréal

PhD - Université de Montréal

Independent visiting researcher - Samsung SAIT

yashgoyal.yg1@gmail.com

Meraj Hashemizadeh

Collaborating researcher - Université de Montréal

Alexia Jolicoeur-Martineau

Independent visiting researcher - Samsung SAIT

Pedram Khorsandi

PhD - Université de Montréal

Independent visiting researcher - Université de Montréal

Sébastien Lachapelle

Independent visiting researcher - Université de Montréal

PhD - Université de Montréal

PhD - Université de Montréal

Co-supervisor :

Ioannis Mitliagkas

PhD - Université de Montréal

Co-supervisor :

Aaron Courville

Rozhin Nobahari

Collaborating Alumni - Université de Montréal

PhD - Université de Montréal

PhD - Université de Montréal

PhD - Université de Montréal

Co-supervisor :

Independent visiting researcher - Samsung SAIT

Motahareh Sohrabi

Collaborating Alumni - Université de Montréal

PhD - Université de Montréal

Independent visiting researcher - Samsung SAIT

Blog Posts

Additive Decoders for Latent Variables Identification and Cartesian-Product Extrapolation

March 18, 2024

Additive Decoders for Latent Variables Identification and Cartesian-Product Extrapolation

by

Sébastien Lachapelle

Ioannis Mitliagkas

Simon Lacoste-Julien

Read the article

Publications

A3T: Adversarially Augmented Adversarial Training

Akram Erraqabi

Aristide Baratin

Simon Lacoste-Julien

Recent research showed that deep neural networks are highly sensitive to so-called adversarial perturbations, which are tiny perturbations o… (see more)f the input data purposely designed to fool a machine learning classifier. Most classification models, including deep learning models, are highly vulnerable to adversarial attacks. In this work, we investigate a procedure to improve adversarial robustness of deep neural networks through enforcing representation invariance. The idea is to train the classifier jointly with a discriminator attached to one of its hidden layer and trained to filter the adversarial noise. We perform preliminary experiments to test the viability of the approach and to compare it to other standard adversarial training methods.

2018-01-12

ArXiv (preprint)

Frank-Wolfe Splitting via Augmented Lagrangian Method

Fabian Pedregosa

Simon Lacoste-Julien

Minimizing a function over an intersection of convex sets is an important task in optimization that is often much more challenging than mini… (see more)mizing it over each individual constraint set. While traditional methods such as Frank-Wolfe (FW) or proximal gradient descent assume access to a linear or quadratic oracle on the intersection, splitting techniques take advantage of the structure of each sets, and only require access to the oracle on the individual constraints. In this work, we develop and analyze the Frank-Wolfe Augmented Lagrangian (FW-AL) algorithm, a method for minimizing a smooth function over convex compact sets related by a "linear consistency" constraint that only requires access to a linear minimization oracle over the individual constraints. It is based on the Augmented Lagrangian Method (ALM), also known as Method of Multipliers, but unlike most existing splitting methods, it only requires access to linear (instead of quadratic) minimization oracles. We use recent advances in the analysis of Frank-Wolfe and the alternating direction method of multipliers algorithms to prove a sublinear convergence rate for FW-AL over general convex compact sets and a linear convergence rate for polytopes.

2018-01-01

AISTATS (published)

proceedings.mlr.press

A Closer Look at Memorization in Deep Networks

Devansh Arpit

Stanisław Jastrzębski

Nicolas Ballas

David Scott Krueger

Emmanuel Bengio

Maxinder S. Kanwal

Asja Fischer

Aaron Courville

Simon Lacoste-Julien

We examine the role of memorization in deep learning, drawing connections to capacity, generalization, and adversarial robustness. While dee… (see more)p networks are capable of memorizing noise data, our results suggest that they tend to prioritize learning simple patterns first. In our experiments, we expose qualitative differences in gradient-based optimization of deep neural networks (DNNs) on noise vs. real data. We also demonstrate that for appropriately tuned explicit regularization (e.g., dropout) we can degrade DNN training performance on noise datasets without compromising generalization on real data. Our analysis suggests that the notions of effective capacity which are dataset independent are unlikely to explain the generalization performance of deep networks when trained with gradient based methods because training data itself plays an important role in determining the degree of memorization.

2017-07-17

Proceedings of the 34th International Conference on Machine Learning (published)

proceedings.mlr.press

A Closer Look at Memorization in Deep Networks

Devansh Arpit

Stanisław Jastrzębski

Nicolas Ballas

David Scott Krueger

Emmanuel Bengio

Maxinder S. Kanwal

Asja Fischer

Aaron Courville

Simon Lacoste-Julien

We examine the role of memorization in deep learning, drawing connections to capacity, generalization, and adversarial robustness. While dee… (see more)p networks are capable of memorizing noise data, our results suggest that they tend to prioritize learning simple patterns first. In our experiments, we expose qualitative differences in gradient-based optimization of deep neural networks (DNNs) on noise vs. real data. We also demonstrate that for appropriately tuned explicit regularization (e.g., dropout) we can degrade DNN training performance on noise datasets without compromising generalization on real data. Our analysis suggests that the notions of effective capacity which are dataset independent are unlikely to explain the generalization performance of deep networks when trained with gradient based methods because training data itself plays an important role in determining the degree of memorization.

2017-06-16

ArXiv (preprint)