Portrait of Nicolas Le Roux

Nicolas Le Roux

Core Industry Member
Adjunct Professor, McGill University, School of Computer Science
Adjunct Professor, Université de Montréal, Department of Computer Science and Operations Research
Research Scientist, Microsoft Research
Research Topics
Deep Learning
Generative Models
Optimization
Reinforcement Learning
Google Scholar

Biography

I am an academic researcher with expertise in machine learning, computer vision, neural networks, deep learning, optimization, large-scale learning and statistical modelling in general.

Current Students

Arnaud Bergeron
Master's Research - Université de Montréal
Co-supervisor :
Joelle Pineau
arnaud.bergeron1@mila.quebec
Jacob Chmura
Master's Research - McGill University
Co-supervisor :
Reihaneh Rabbany
jacob.chmura@mila.quebec
Website
Github
Raesetje Sefala
PhD - McGill University
Principal supervisor :
Joelle Pineau
raesetje.sefala@mila.quebec
Website
Google Scholar

Publications

On the interplay between noise and curvature and its effect on optimization and generalization
Valentin Thomas
Fabian Pedregosa
Bart Van Merriënboer
Pierre-Antoine Mangazol
Yoshua Bengio
Nicolas Le Roux
The speed at which one can minimize an expected loss using stochastic methods depends on two properties: the curvature of the loss and the v… (see more)ariance of the gradients. While most previous works focus on one or the other of these properties, we explore how their interaction affects optimization speed. Further, as the ultimate goal is good generalization performance, we clarify how both curvature and noise are relevant to properly estimate the generalization gap. Realizing that the limitations of some existing works stems from a confusion between these matrices, we also clarify the distinction between the Fisher matrix, the Hessian, and the covariance matrix of the gradients.
2019-12-31
AISTATS (published)
doi.org
proceedings.mlr.press