Portrait of Ross Goroshin

Ross Goroshin

Core Industry Member
Adjunct professor, Université de Montréal, Department of Computer Science and Operations Research
Google DeepMind
Research Topics
Applied AI
Computer Vision
Deep Learning
Dynamical Systems
Representation Learning

Biography

Ross Goroshin is a Research Scientist at Google DeepMind, Montreal and Core Industry Member at Mila - Quebec Artificial Intelligence Institute. He holds a PhD in Computer Science from NYU, where he was advised by Yann LeCun. He also earned a B.Eng. in Electrical Engineering from Concordia University and an M.S. in Electrical Engineering from Georgia Tech. His research focuses on computer vision, self-supervised learning, and optimal control.

In addition to his roles at Google DeepMind and Mila, Ross serves as an adjunct professor in the Department of Computer Science and Operations Research (DIRO) at Université de Montréal.

Current Students

Kanishk Jain
PhD - Université de Montréal
Principal supervisor :
Sarath Chandar
kanishk.jain@mila.quebec
Website
Github
Google Scholar
Aidan Li
Master's Research - Université de Montréal
Principal supervisor :
Sarath Chandar
aidan.li@mila.quebec
Website
Github
LinkedIn
Google Scholar

Publications

Comparing Transfer and Meta Learning Approaches on a Unified Few-Shot Classification Benchmark
Vincent Dumoulin
Neil Houlsby
Utku Evci
Xiaohua Zhai
Ross Goroshin
Sylvain Gelly
Hugo Larochelle
Meta and transfer learning are two successful families of approaches to few-shot learning. Despite highly related goals, state-of-the-art ad… (see more)vances in each family are measured largely in isolation of each other. As a result of diverging evaluation norms, a direct or thorough comparison of different approaches is challenging. To bridge this gap, we perform a cross-family study of the best transfer and meta learners on both a large-scale meta-learning benchmark (Meta-Dataset, MD), and a transfer learning benchmark (Visual Task Adaptation Benchmark, VTAB). We find that, on average, large-scale transfer methods (Big Transfer, BiT) outperform competing approaches on MD, even when trained only on ImageNet. In contrast, meta-learning approaches struggle to compete on VTAB when trained and validated on MD. However, BiT is not without limitations, and pushing for scale does not improve performance on highly out-of-distribution MD tasks. In performing this study, we reveal a number of discrepancies in evaluation norms and study some of these in light of the performance gap. We hope that this work facilitates sharing of insights from each community, and accelerates progress on few-shot learning.
2021-04-05
ArXiv (preprint)
arxiv.org
arxiv.org
An Effective Anti-Aliasing Approach for Residual Networks
Cristina Vasconcelos
Hugo Larochelle
Vincent Dumoulin
Nicolas Roux
Ross Goroshin
Image pre-processing in the frequency domain has traditionally played a vital role in computer vision and was even part of the standard pipe… (see more)line in the early days of deep learning. However, with the advent of large datasets, many practitioners concluded that this was unnecessary due to the belief that these priors can be learned from the data itself. Frequency aliasing is a phenomenon that may occur when sub-sampling any signal, such as an image or feature map, causing distortion in the sub-sampled output. We show that we can mitigate this effect by placing non-trainable blur filters and using smooth activation functions at key locations, particularly where networks lack the capacity to learn them. These simple architectural changes lead to substantial improvements in out-of-distribution generalization on both image classification under natural corruptions on ImageNet-C [10] and few-shot learning on Meta-Dataset [17], without introducing additional trainable parameters and using the default hyper-parameters of open source codebases.
2020-11-19
ArXiv (preprint)
arxiv.org
arxiv.org