Aaron Courville

sarvjeet-singh.ghotra@mila.quebec

Juan Duque

PhD - Université de Montréal

juan.duque@mila.quebec

PhD - Université de Montréal

udaykaran.kapur@mila.quebec

Arian Hosseini

PhD - Université de Montréal

Uday Kapur

PhD - Université de Montréal

Amr Khalifa

PhD - Université de Montréal

amr-khalifa.alshaykh@mila.quebec

alan.milligan@mila.quebec

Zhixuan Lin

PhD - Université de Montréal

PhD - Université de Montréal

Co-supervisor :

Pablo Samuel Castro

waltermayor@gmail.com

PhD - Université de Montréal

Principal supervisor :

Simon Lacoste-Julien

PhD - Université de Montréal

Principal supervisor :

Gauthier Gidel

andjela.mladenovic@mila.quebec

mohammed.muqeeth@mila.quebec

Muqeeth Mohammed

PhD - Université de Montréal

PhD - Université de Montréal

Co-supervisor :

Rishabh Agarwal

morgane.moss@mila.quebec

Andrei Nicolicioiu

PhD - Université de Montréal

andrei.nicolicioiu@gmail.com

michell.payanoperez@mila.quebec

Michael Noukhovitch

PhD - Université de Montréal

Johan Samir Obando Ceron

PhD - Université de Montréal

Co-supervisor :

Pablo Samuel Castro

johan.ceron@mila.quebec

PhD - Université de Montréal

Dereck Piché

PhD - Université de Montréal

pichedereck@gmail.com

khaled.rouissi@mila.quebec

Khaled Rouissi

Master's Research - Université de Montréal

Esra'a Saleh

PhD - Université de Montréal

Principal supervisor :

Glen Berseth

esraa.saleh@mila.quebec

Vedant Shah

PhD - Université de Montréal

vedant.shah@mila.quebec

colin.yiyang-lu@mila.quebec

Yusong Wu

PhD - Université de Montréal

Principal supervisor :

Anna (Cheng-Zhi) Huang

wu.yusong@mila.quebec

Colin Yiyang Lu

PhD - Université de Montréal

Sujin yun

PhD - Université de Montréal

sujin.yun@mila.quebec

xiaofeng.zhang@mila.quebec

Xiaofeng Zhang

PhD - Université de Montréal

Publications

Explicitly Modeling Syntax in Language Models with Incremental Parsing and a Dynamic Oracle

Syntax is fundamental to our thinking about language. Failing to capture the structure of input language could lead to generalization proble… (see more)ms and over-parametrization. In the present work, we propose a new syntax-aware language model: Syntactic Ordered Memory (SOM). The model explicitly models the structure with an incremental parser and maintains the conditional probability setting of a standard language model (left-to-right). To train the incremental parser and avoid exposure bias, we also propose a novel dynamic oracle, so that SOM is more robust to wrong parsing decisions. Experiments show that SOM can achieve strong results in language modeling, incremental parsing and syntactic generalization tests, while using fewer parameters than other models.

2021-05-31

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (published)

arxiv.org

Data-Efficient Reinforcement Learning with Self-Predictive Representations

Max Schwarzer

Ankesh Anand

Rishab Goel

R Devon Hjelm

Philip Bachman

While deep reinforcement learning excels at solving tasks where large amounts of data can be collected through virtually unlimited interacti… (see more)on with the environment, learning from limited interaction remains a key challenge. We posit that an agent can learn more efficiently if we augment reward maximization with self-supervised objectives based on structure in its visual input and sequential interaction with the environment. Our method, Self-Predictive Representations(SPR), trains an agent to predict its own latent state representations multiple steps into the future. We compute target representations for future states using an encoder which is an exponential moving average of the agent's parameters and we make predictions using a learned transition model. On its own, this future prediction objective outperforms prior methods for sample-efficient deep RL from pixels. We further improve performance by adding data augmentation to the future prediction loss, which forces the agent's representations to be consistent across multiple views of an observation. Our full self-supervised objective, which combines future prediction and data augmentation, achieves a median human-normalized score of 0.415 on Atari in a setting limited to 100k steps of environment interaction, which represents a 55% relative improvement over the previous state-of-the-art. Notably, even in this limited data regime, SPR exceeds expert human scores on 7 out of 26 games. The code associated with this work is available at https://github.com/mila-iqia/spr

2021-05-30

Université de Montréal (published)

Convex Potential Flows: Universal Probability Distributions with Optimal Transport and Convex Optimization

Chin-wei Huang

Ricky T. Q. Chen

Christos Tsirigotis

Flow-based models are powerful tools for designing probabilistic models with tractable density. This paper introduces Convex Potential Flows… (see more) (CP-Flow), a natural and efficient parameterization of invertible models inspired by the optimal transport (OT) theory. CP-Flows are the gradient map of a strongly convex neural potential function. The convexity implies invertibility and allows us to resort to convex optimization to solve the convex conjugate for efficient inversion. To enable maximum likelihood training, we derive a new gradient estimator of the log-determinant of the Jacobian, which involves solving an inverse-Hessian vector product using the conjugate gradient method. The gradient estimator has constant-memory cost, and can be made effectively unbiased by reducing the error tolerance level of the convex optimization routine. Theoretically, we prove that CP-Flows are universal density approximators and are optimal in the OT sense. Our empirical results show that CP-Flow performs competitively on standard benchmarks of density estimation and variational inference.

2021-05-02

International Conference on Learning Representations (Poster)

Integrating Categorical Semantics into Unsupervised Domain Translation

Samuel Lavoie

Faruk Ahmed

While unsupervised domain translation (UDT) has seen a lot of success recently, we argue that mediating its translation via categorical sema… (see more)ntic features could broaden its applicability. In particular, we demonstrate that categorical semantics improves the translation between perceptually different domains sharing multiple object categories. We propose a method to learn, in an unsupervised manner, categorical semantic features (such as object labels) that are invariant of the source and target domains. We show that conditioning the style encoder of unsupervised domain translation methods on the learned categorical semantics leads to a translation preserving the digits on MNIST

2021-05-02

International Conference on Learning Representations (Poster)

Iterated Learning for Emergent Systematicity in VQA

Ankit Vani

Max Schwarzer

Yuchen Lu

Eeshan Dhekane

Although neural module networks have an architectural bias towards compositionality, they require gold standard layouts to generalize system… (see more)atically in practice. When instead learning layouts and modules jointly, compositionality does not arise automatically and an explicit pressure is necessary for the emergence of layouts exhibiting the right structure. We propose to address this problem using iterated learning, a cognitive science theory of the emergence of compositional languages in nature that has primarily been applied to simple referential games in machine learning. Considering the layouts of module networks as samples from an emergent language, we use iterated learning to encourage the development of structure within this language. We show that the resulting layouts support systematic generalization in neural agents solving the more complex task of visual question-answering. Our regularized iterated learning method can outperform baselines without iterated learning on SHAPES-SyGeT (SHAPES Systematic Generalization Test), a new split of the SHAPES dataset we introduce to evaluate systematic generalization, and on CLOSURE, an extension of CLEVR also designed to test systematic generalization. We demonstrate superior performance in recovering ground-truth compositional program structure with limited supervision on both SHAPES-SyGeT and CLEVR.

2021-05-02

International Conference on Learning Representations (Oral)

Learning Task Decomposition with Ordered Memory Policy Network

Yuchen Lu

Yikang Shen

Siyuan Zhou

Joshua B. Tenenbaum

Chuang Gan

Many complex real-world tasks are composed of several levels of sub-tasks. Humans leverage these hierarchical structures to accelerate the l… (see more)earning process and achieve better generalization. In this work, we study the inductive bias and propose Ordered Memory Policy Network (OMPN) to discover subtask hierarchy by learning from demonstration. The discovered subtask hierarchy could be used to perform task decomposition, recovering the subtask boundaries in an unstruc-tured demonstration. Experiments on Craft and Dial demonstrate that our modelcan achieve higher task decomposition performance under both unsupervised and weakly supervised settings, comparing with strong baselines. OMPN can also bedirectly applied to partially observable environments and still achieve higher task decomposition performance. Our visualization further confirms that the subtask hierarchy can emerge in our model.

2021-05-02

International Conference on Learning Representations (Poster)

Neural Approximate Sufficient Statistics for Implicit Models

Yanzhi Chen

Dinghuai Zhang

Michael U. Gutmann

Zhanxing Zhu

We consider the fundamental problem of how to automatically construct summary statistics for implicit generative models where the evaluation… (see more) of the likelihood function is intractable, but sampling data from the model is possible. The idea is to frame the task of constructing sufficient statistics as learning mutual information maximizing representations of the data with the help of deep neural networks. The infomax learning procedure does not need to estimate any density or density ratio. We apply our approach to both traditional approximate Bayesian computation and recent neural likelihood methods, boosting their performance on a range of tasks.

2021-05-02

International Conference on Learning Representations (Spotlight)

Systematic Generalisation with Group Invariant Predictions

Faruk Ahmed

Yoshua Bengio

Harm van Seijen

2021-05-02

International Conference on Learning Representations (Spotlight)

Touch-based Curiosity for Sparse-Reward Tasks

Sai Rajeswar

Cyril Ibrahim

Nitin Surya

Florian Golemo

David Vázquez

Pedro O. Pinheiro

2021-03-31

ArXiv (preprint)

arxiv.org

Pretraining Reward-Free Representations for Data-Efficient Reinforcement Learning

Philip Bachman

2021-03-08

International Conference on Learning Representations (unknown)

DATA-EFFICIENT REINFORCEMENT LEARNING

R Devon Hjelm

Philip Bachman

Data efficiency poses a major challenge for deep reinforcement learning. We approach this issue from the perspective of self-supervised repr… (see more)esentation learning, leveraging reward-free exploratory data to pretrain encoder networks. We employ a novel combination of latent dynamics modelling and goal-reaching objectives, which exploit the inherent structure of data in reinforcement learning. We demonstrate that our method scales well with network capacity and pretraining data. When evaluated on the Atari 100k data-efficiency benchmark, our approach significantly outperforms previous methods combining unsupervised pretraining with task-specific finetuning, and approaches human-level performance.

2020-12-31

(published)

www.semanticscholar.org

Emergent Communication under Competition

Michael Noukhovitch

Travis LaCroix

Angeliki Lazaridou

2020-12-31

AAMAS (published)