Portrait of Ioannis Mitliagkas

Ioannis Mitliagkas

Core Academic Member
ioannis@mila.quebec
Canada CIFAR AI Chair
Associate Professor, Université de Montréal, Department of Computer Science and Operations Research
Research Scientist, Google DeepMind
Research Topics
Deep Learning
Distributed Systems
Dynamical Systems
Generative Models
Machine Learning Theory
Optimization
Representation Learning
Website
Google Scholar
LinkedIn
X

Biography

Ioannis Mitliagkas (Γιάννης Μητλιάγκας) is an associate professor in the Department of Computer Science and Operations Research (DIRO) at Université de Montréal, as well as a Core Academic member of Mila – Quebec Artificial Intelligence Institute and a Canada CIFAR AI Chair. He holds a part-time position as a staff research scientist at Google DeepMind Montréal.

Previously, he was a postdoctoral scholar in the Departments of statistics and computer science at Stanford University. He obtained his PhD from the Department of Electrical and Computer Engineering at the University of Texas at Austin.

His research includes topics in machine learning, with emphasis on optimization, deep learning theory, statistical learning. His recent work includes methods for efficient and adaptive optimization, studying the interaction between optimization and the dynamics of large-scale learning systems and the dynamics of games.

Current Students

Mehdi Inane Ahmed
PhD - Université de Montréal
Github
Google Scholar
Ryan D'Orazio
PhD - Université de Montréal
Website
Google Scholar
Charles Guille-escuret
Collaborating Alumni - Université de Montréal
Website
Github
Google Scholar
Adam Ibrahim
Collaborating Alumni - Université de Montréal
Co-supervisor :
Irina Rish
Website
Google Scholar
Zichu Liu
PhD - Université de Montréal
Principal supervisor :
Gauthier Gidel
Youry Macius
Professional Master's - Université de Montréal
youry.macius@umontreal.ca
Website
Github
Google Scholar
Divyat Mahajan
PhD - Université de Montréal
Yukti Makhija
PhD - Université de Montréal
Principal supervisor :
Simon Lacoste-Julien
Website
Github
Google Scholar
Hiroki Naganuma
PhD - Université de Montréal
Website
Github
Google Scholar
Vincent Quirion
Master's Research - Université de Montréal
Github

Publications

Accelerated Stochastic Power Iteration
Peng Xu
Bryan Dawei He
Christopher De Sa
Ioannis Mitliagkas
Christopher Re
Principal component analysis (PCA) is one of the most powerful tools in machine learning. The simplest method for PCA, the power iteration, … (see more)requires O ( 1 / Δ ) full-data passes to recover the principal component of a matrix with eigen-gap Δ. Lanczos, a significantly more complex method, achieves an accelerated rate of O ( 1 / Δ ) passes. Modern applications, however, motivate methods that only ingest a subset of available data, known as the stochastic setting. In the online stochastic setting, simple algorithms like Oja's iteration achieve the optimal sample complexity O ( σ 2 / Δ 2 ) . Unfortunately, they are fully sequential, and also require O ( σ 2 / Δ 2 ) iterations, far from the O ( 1 / Δ ) rate of Lanczos. We propose a simple variant of the power iteration with an added momentum term, that achieves both the optimal sample and iteration complexity. In the full-pass setting, standard analysis shows that momentum achieves the accelerated rate, O ( 1 / Δ ) . We demonstrate empirically that naively applying momentum to a stochastic method, does not result in acceleration. We perform a novel, tight variance analysis that reveals the "breaking-point variance" beyond which this acceleration does not occur. By combining this insight with modern variance reduction techniques, we construct stochastic PCA algorithms, for the online and offline setting, that achieve an accelerated iteration complexity O ( 1 / Δ ) . Due to the embarassingly parallel nature of our methods, this acceleration translates directly to wall-clock time if deployed in a parallel environment. Our approach is very general, and applies to many non-convex optimization problems that can now be accelerated using the same technique.
2017-07-10
ArXiv (preprint)
arxiv.org
arxiv.org