Mila members to stand out at ICML 2021

Next month, Mila researchers will showcase their work in full force at the 38th International Conference on Machine Learning (ICML). This annual event brings together some of the brightest minds of the machine learning research community and will run virtually from July 18 to 24.

Of the 5,513 submissions this year, 1,184 papers were accepted (21.5%); 30 of those papers were co-authored by Mila researchers. 

The work being presented by Mila researchers covers a wide range of topics and demonstrates the far-reaching potential applications of machine learning in areas such as health, chemical physics, and even the airline industry. 

In “Structured Convolutional Kernel Networks for Airline Crew Scheduling,” Mila core academic member Simon Lacoste-Julien and colleagues introduce a novel deep structured predictor: structured convolutional kernel networks (Struct-CKN). The initial solutions obtained using Struct-CKN on a flight-connection dataset revealed that it can be further refined by an airline crew scheduling solver. 

Core faculty member Jian Tang and team designed a unified framework, Local-instance and Global-semantic Learning (GraphLoG), for self-supervised whole-graph representation learning, proposing hierarchical prototypes to capture global semantic clusters. The verified experiments on chemical and biological benchmark data sets demonstrate the method’s effectiveness, with potential implications for many tasks such as molecule properties prediction in drug and material discovery. 

On the other end of the spectrum, in “Continuous Coordination As a Realistic Scenario for Lifelong Learning,” Mila University professors Sarath Chandar and Aaron Courville, as well as research intern Hadi Nekoei and Master’s student Akilesh Badrinaaraayanan introduce Lifelong Hanabi, a continual multi-agent reinforcement learning testbed in which every task is coordinating with a partner that's an expert player of Hanabi.

Below, accepted papers (spotlight and oral), workshops, and tutorials co-authored and co-organized by Mila members:

Tutorials

Continual learning with deep architectures

Vincenzo Lomonaco, Irina Rish

https://icml.cc/Conferences/2021/Schedule?showEvent=10833

Random Matrix Theory and ML (RMT+ML)

Fabian Pedregosa, Jeffrey Pennington, Thomas Trogdon, Courtney Paquette

https://icml.cc/Conferences/2021/Schedule?showEvent=10840

Oral Presentations

Directional Graph Networks

Dominique Beaini, Saro Passaro, Vincent Létourneau, Will Hamilton, Gabriele Corso, Pietro Lió

https://arxiv.org/abs/2010.02863

Can Subnetwork Structure Be the Key to Out-of-Distribution Generalization?

Dinghuai Zhang, Kartik Ahuja, Yilun Xu, Yisen Wang, Aaron Courville

https://arxiv.org/pdf/2106.02890.pdf

Out-of-Distribution Generalization via Risk Extrapolation (REx)

David Krueger, Ethan Caballero, Joern-Henrik Jacobsen, Amy Zhang, Jonathan Binas, Dinghuai Zhang, Rémi Le Priol, Aaron Courville

https://arxiv.org/abs/2003.00688

Learning Gradient Fields for Molecular Conformation Generation

Chence Shi, Shitong Luo, Minkai Xu, Jian Tang

https://arxiv.org/abs/2105.03902

On Disentangled Representations Learned from Correlated Data

Frederik Träuble, Elliot Creager, Niki Kilbertus, Francesco Locatello, Andrea Dittadi, Anirudh Goyal, Bernhard Schölkopf, Stefan Bauer

https://arxiv.org/abs/2006.07886

RNN with Particle Flow for Probabilistic Spatio-temporal Forecasting

Soumyasundar Pal, Liheng Ma, Yingxue Zhang, Mark Coates 

https://arxiv.org/abs/2106.06064

Spotlight Presentations

Aggregating from Multiple Target-Shifted Sources

Changjian Shui, Zijian Li, Jiaqi Li, Christian Gagné, Charles X. Ling, Boyu Wang

https://arxiv.org/abs/2105.04051

Continuous Coordination As a Realistic Scenario For Lifelong Learning

Hadi Nekoei, Akilesh Badrinaaraayanan, Aaron Courville, Sarath Chandar

https://arxiv.org/abs/2103.03216

Diffusion Earth Mover’s Distance and Distribution Embeddings

Alexander Tong, Guillaume Huguet, Amine Natik, Kincaid MacDonald, Manik Kuchroo, Ronald Coifman, Guy Wolf, Smita Krishnaswamy

https://arxiv.org/abs/2102.12833

Learning a Universal Template for Few-shot Dataset Generalization

Eleni Triantafillou, Hugo Larochelle, Richard Zemel, Vincent Dumoulin

https://arxiv.org/abs/2105.07029

Self-supervised Graph-level Representation Learning with Local and Global Structure

Minghao Xu, Hang Wang, Bingbing Ni, Hongyu Guo, Jian Tang

https://arxiv.org/pdf/2106.04113.pdf

Affine Invariant Analysis of Frank-Wolfe on Strongly Convex Sets

Thomas Kerdreux, Lewis Liu, Simon Lacoste-Julien, Damien Scieur

https://arxiv.org/abs/2011.03351

Infinite-Dimensional Optimization for Zero-Sum Games via Variational Transport

Lewis Liu, Yufeng Zhang, Zhuoran Yang, Reza Babanezhad, Zhaoran Wang

https://opt-ml.org/papers/2020/paper_92.pdf

Simultaneous Similarity-based Self-Distillation for Deep Metric Learning

Karsten Roth, Timo Milbich, Bjorn Ommer, Joseph Paul Cohen, Marzyeh Ghassemi

https://arxiv.org/abs/2009.08348

Structured Convolutional Kernel Networks for Airline Crew Scheduling

Yassine Yaakoubi, François Soumis, Simon Lacoste-Julien 

https://arxiv.org/abs/2105.11646

An End-to-End Framework for Molecular Conformation Generation via Bilevel Programming

Minkai Xu, Wujie Wang, Shitong Luo, Chence Shi, Yoshua Bengio, Rafael Gomez-Bombarelli, Jian Tang

https://arxiv.org/abs/2105.07246

Catastrophic Fisher Explosion: Early Phase Fisher Matrix Impacts Generalization

Stanislaw Jastrzebski, Devansh Arpit, Oliver Astrand, Giancarlo Kerg, Huan Wang, Caiming Xiong, Richard Socher, Kyunghyun Cho, Krzysztof J Geras

https://arxiv.org/abs/2012.14193

Preferential Temporal Difference Learning

Nishanth Anand, Doina Precup

https://arxiv.org/abs/2106.06508

Exploration in Approximate Hyper-State Space for Meta Reinforcement Learning

Luisa Zintgraf, Leo Feng, Cong Lu, Maximilian Igl, Kristian Hartikainen, Katja Hofmann, Shimon Whiteson

https://arxiv.org/abs/2010.01062

Equivariant Networks for Pixelized Spheres

Mehran Shakerinava, Siamak Ravanbakhsh

OptiDICE: Offline Policy Optimization via Stationary Distribution Correction Estimation

Jongmin Lee, Wonseok Jeon, Byung-Jun Lee, Joelle Pineau, Kee-Eung Kim

Beyond Variance Reduction: Understanding the True Impact of Baselines on Policy Optimization

Wesley Chung, Valentin Thomas, Marlos C. Machado, Nicolas Le Roux 

https://arxiv.org/abs/2008.13773

Randomized Exploration in Reinforcement Learning with General Value Function Approximation

Haque Ishfaq, Qiwen Cui, Alex Ayoub, Viet Nguyen, Zhuoran Yang, Zhaoran Wang, Doina Precup, Lin Yang

Locally Persistent Exploration in Continuous Control Tasks with Sparse Rewards

Susan Amin, Maziar Gomrokchi, Hossein Aboutalebi, Harsh Satija, Doina Precup 

https://arxiv.org/abs/2012.13658

Non-Autoregressive Electron Redistribution Modeling for Reaction Prediction

Hangrui Bi, Hengyi Wang, Chence Shi, Connor Coley, Jian Tang, Hongyu Guo

https://icml.cc/Conferences/2021/ScheduleMultitrack?event=9427

Trajectory Diversity for Zero-Shot Coordination

Andrei Lupu, Brandon Cui, Hengyuan Hu, Jakob Foerster

https://icml.cc/Conferences/2021/ScheduleMultitrack?event=9434

Multi-Task Reinforcement Learning with Context-based Representations

Shagun Sodhani, Amy Zhang, Joelle Pineau

https://arxiv.org/pdf/2102.06177.pdf

Robust Representation Learning via Perceptual Similarity Metrics

Saeid A Taghanaki, Kristy Choi, Amir Hosein Khasahmadi, Anirudh Goyal 

Vector Quantized Models for Planning

Sherjil Ozair, Yazhe Li, Ali Razavi, Ioannis Antonoglou, Aäron van den Oord, Oriol Vinyals 

https://arxiv.org/abs/2106.04615

Efficient Deviation Types and Learning for Hindsight Rationality in Extensive-Form Games

Dustin Morrill, Ryan D'Orazio, Marc Lanctot, James Wright, Michael Bowling, Amy Greenwald 

https://arxiv.org/abs/2102.06973

A Deep Reinforcement Learning Approach to Marginalized Importance Sampling with the Successor Representation

Scott Fujimoto, David Meger, Doina Precup

Workshops

Tackling Climate Change with Machine Learning

Hari Prasanna Das, Meareg Hailemariam, Katarzyna Tokarska, Maria João Sousa, David Rolnick, Xiaoxiang Zhu, Yoshua Bengio

Workshop Home Page

Unsupervised Reinforcement Learning

Feryal Behbahani, Joelle Pineau, Lerrel Pinto, Roberta Raileanu, Aravind Srinivas, Denis Yarats, Amy Zhang

Workshop Home Page

INNF+: Invertible Neural Networks, Normalizing Flows, and Explicit Likelihood Models

Chin-Wei Huang, David Krueger, Rianne Van den Berg, George Papamakarios, Tian Qi Chen, Danilo J. Rezende

Workshop Home Page

Theory and Foundation of Continual Learning

Thang Doan, Bogdan Mazoure, Amal Rannen Triki, Rahaf Aljundi, Vincenzo Lomonaco, Xu He, Arslan Chaudhry

Workshop Home Page

Women in Machine Learning virtual Un-workshop

Wenshuo Guo, Amy Zhang, Nezihe Merve Gürel, Audrey Durand, Ehi Nosakhare, Christina Papadimitriou, Sarah Poole, Tatjana Chavdarova

Workshop Home Page