Aaron Courville

alan.milligan@mila.quebec

Alan Alan

Doctorat - UdeM

Superviseur⋅e principal⋅e :

Simon Lacoste-Julien

Anirudh Buvanesh

Doctorat - UdeM

Superviseur⋅e principal⋅e :

Laurent Charlin

anirudb1102@gmail.com

sarvjeet-singh.ghotra@mila.quebec

Juan Duque

Doctorat - UdeM

juan.duque@mila.quebec

Doctorat - UdeM

Doctorat - UdeM

Uday Kapur

Doctorat - UdeM

udaykaran.kapur@mila.quebec

Amr Khalifa

Doctorat - UdeM

amr-khalifa.alshaykh@mila.quebec

andjela.mladenovic@mila.quebec

Zhixuan Lin

Doctorat - UdeM

Doctorat - UdeM

Co-superviseur⋅e :

Pablo Samuel Castro

waltermayor@gmail.com

Doctorat - UdeM

Superviseur⋅e principal⋅e :

Gauthier Gidel

mohammed.muqeeth@mila.quebec

Muqeeth Mohammed

Doctorat - UdeM

Doctorat - UdeM

Co-superviseur⋅e :

Rishabh Agarwal

morgane.moss@mila.quebec

Andrei Nicolicioiu

Doctorat - UdeM

andrei.nicolicioiu@gmail.com

Doctorat - UdeM

Johan Samir Obando Ceron

Doctorat - UdeM

Co-superviseur⋅e :

Pablo Samuel Castro

johan.ceron@mila.quebec

Doctorat - UdeM

michell.payanoperez@mila.quebec

Dereck Piché

Doctorat - UdeM

pichedereck@gmail.com

khaled.rouissi@mila.quebec

Khaled Rouissi

Maîtrise recherche - UdeM

Esra'a Saleh

Doctorat - UdeM

Superviseur⋅e principal⋅e :

Glen Berseth

esraa.saleh@mila.quebec

Vedant Shah

Doctorat - UdeM

vedant.shah@mila.quebec

colin.yiyang-lu@mila.quebec

Yusong Wu

Doctorat - UdeM

Superviseur⋅e principal⋅e :

Anna (Cheng-Zhi) Huang

wu.yusong@mila.quebec

Colin Yiyang Lu

Doctorat - UdeM

Sujin yun

Doctorat - UdeM

sujin.yun@mila.quebec

xiaofeng.zhang@mila.quebec

Xiaofeng Zhang

Doctorat - UdeM

Publications

Recursive Top-Down Production for Sentence Generation with Latent Trees

Timothy J. O'Donnell

We model the recursive production property of context-free grammars for natural and synthetic languages. To this end, we present a dynamic p… (voir plus)rogramming algorithm that marginalises over latent binary tree structures with

2020-10-31

Findings of the Association for Computational Linguistics: EMNLP 2020 (publié)

Supervised Seeded Iterated Learning for Interactive Language Learning

Yuchen Lu

Soumye Singhal

Florian Strub

Olivier Pietquin

2020-10-31

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) (publié)

NU-GAN: High resolution neural upsampling with GAN

In this paper, we propose NU-GAN, a new method for resampling audio from lower to higher sampling rates (upsampling). Audio upsampling is an… (voir plus) important problem since productionizing generative speech technology requires operating at high sampling rates. Such applications use audio at a resolution of 44.1 kHz or 48 kHz, whereas current speech synthesis methods are equipped to handle a maximum of 24 kHz resolution. NU-GAN takes a leap towards solving audio upsampling as a separate component in the text-to-speech (TTS) pipeline by leveraging techniques for audio generation using GANs. ABX preference tests indicate that our NU-GAN resampler is capable of resampling 22 kHz to 44.1 kHz audio that is distinguishable from original audio only 7.4% higher than random chance for single speaker dataset, and 10.8% higher than chance for multi-speaker dataset.

2020-10-21

arXiv (prépublication)

Explicitly Modeling Syntax in Language Model improves Generalization

Syntax is fundamental to our thinking about language. Although neural networks are very successful in many tasks, they do not explicitly mod… (voir plus)el syntactic structure. Failing to capture the structure of inputs could lead to generalization problems and over-parametrization. In the present work, we propose a new syntax-aware language model: Syntactic Ordered Memory (SOM). The model explicitly models the structure with a one-step look-ahead parser and maintains the conditional probability setting of the standard language model. Experiments show that SOM can achieve strong results in language modeling and syntactic generalization tests, while using fewer parameters then other models.

2020-10-20

arXiv.org (prépublication)

dblp.uni-trier.de

Probability Distillation: A Caveat and Alternatives

Alexandre Lacoste

Due to Van den Oord et al. (2018), probability distillation has recently been of interest to deep learning practitioners, where, as a practi… (voir plus)cal workaround for deploying autoregressive models in real-time applications, a student network is used to obtain quality samples in parallel. We identify a pathological optimization issue with the adopted stochastic minimization of the reverse-KL divergence: the curse of dimensionality results in a skewed gradient distribution that renders training inefﬁcient. This means that KL-based “evaluative” training can be susceptible to poor exploration if the target distribution is highly structured. We then explore alternative principles for distillation, including one with an “instructive” signal, and show that it is possible to achieve qualitatively better results than with KL minimization.

2020-08-05

Uncertainty in Artificial Intelligence (publié)

proceedings.mlr.press

A Large-Scale, Open-Domain, Mixed-Interface Dialogue-Based ITS for STEM

Iulian V. Serban

Varun Gupta

Ekaterina Kochmar

Dung D. Vu

Robert Belfer

2020-06-09

Artificial Intelligence in Education (publié)

Gintare Karolina Dziugaite

Stochastic Neural Network with Kronecker Flow

Chin-wei Huang

Ahmed Touati

Pascal Vincent

Alexandre Lacoste

Recent advances in variational inference enable the modelling of highly structured joint distributions, but are limited in their capacity to… (voir plus) scale to the high-dimensional setting of stochastic neural networks. This limitation motivates a need for scalable parameterizations of the noise generation process, in a manner that adequately captures the dependencies among the various parameters. In this work, we address this need and present the Kronecker Flow, a generalization of the Kronecker product to invertible mappings designed for stochastic neural networks. We apply our method to variational Bayesian neural networks on predictive tasks, PAC-Bayes generalization bound estimation, and approximate Thompson sampling in contextual bandits. In all setups, our methods prove to be competitive with existing methods and better than the baselines.

2020-06-02

Artificial Intelligence and Statistics (publié)

proceedings.mlr.press

Detecting semantic anomalies

Faruk Ahmed

We critically appraise the recent interest in out-of-distribution (OOD) detection and question the practical relevance of existing benchmark… (voir plus)s. While the currently prevalent trend is to consider different datasets as OOD, we argue that out-distributions of practical interest are ones where the distinction is semantic in nature for a specified context, and that evaluative tasks should reflect this more closely. Assuming a context of object recognition, we recommend a set of benchmarks, motivated by practical applications. We make progress on these benchmarks by exploring a multi-task learning based approach, showing that auxiliary objectives for improved semantic awareness result in improved semantic anomaly detection, with accompanying generalization benefits.

2020-04-02

Proceedings of the AAAI Conference on Artificial Intelligence (publié)

Pix2Shape – Towards Unsupervised Learning of 3D Scenes from Images using a View-based Representation

Sai Rajeswar

Fahim Mannan

Florian Golemo

Jérôme Parent-lévesque

David Vázquez

Derek Nowrouzezahrai

We infer and generate three-dimensional (3D) scene information from a single input image and without supervision. This problem is under-expl… (voir plus)ored, with most prior work relying on supervision from, e.g., 3D ground-truth, multiple images of a scene, image silhouettes or key-points. We propose Pix2Shape, an approach to solve this problem with four components: (i) an encoder that infers the latent 3D representation from an image, (ii) a decoder that generates an explicit 2.5D surfel-based reconstruction of a scene from the latent code (iii) a differentiable renderer that synthesizes a 2D image from the surfel representation, and (iv) a critic network trained to discriminate between images generated by the decoder-renderer and those from a training distribution. Pix2Shape can generate complex 3D scenes that scale with the view-dependent on-screen resolution, unlike representations that capture world-space resolution, i.e., voxels or meshes. We show that Pix2Shape learns a consistent scene representation in its encoded latent space and that the decoder can then be applied to this latent representation in order to synthesize the scene from a novel viewpoint. We evaluate Pix2Shape with experiments on the ShapeNet dataset as well as on a novel benchmark we developed, called 3D-IQTT, to evaluate models based on their ability to enable 3d spatial reasoning. Qualitative and quantitative evaluation demonstrate Pix2Shape's ability to solve scene reconstruction, generation, and understanding tasks.

2020-03-19

International Journal of Computer Vision (inconnu)

Solving ODE with Universal Flows: Approximation Theory for Flow-Based Models

Chin-wei Huang

Laurent Dinh

Normalizing flows are powerful invertible probabilistic models that can be used to translate two probability distributions, in a way that al… (voir plus)lows us to efficiently track the change of probability density. However, to trade for computational efficiency in sampling and in evaluating the log-density, special parameterization designs have been proposed at the cost of representational expressiveness. In this work, we propose to use ODEs as a framework to establish universal approximation theory for certain families of flow-based models.

2020-02-25

International Conference on Learning Representations (publié)

openreview.net

Augmented Normalizing Flows: Bridging the Gap Between Generative Flows and Latent Variable Models

Chin-wei Huang

Laurent Dinh

In this work, we propose a new family of generative flows on an augmented data space, with an aim to improve expressivity without drasticall… (voir plus)y increasing the computational cost of sampling and evaluation of a lower bound on the likelihood. Theoretically, we prove the proposed flow can approximate a Hamiltonian ODE as a universal transport map. Empirically, we demonstrate state-of-the-art performance on standard benchmarks of flow-based generative modeling.

2020-02-16

ArXiv (prépublication)

On Bonus-Based Exploration Methods in the Arcade Learning Environment

Adrien Ali Taiga

William Fedus

Marlos C. Machado

Bellemare Marc-Emmanuel

Research on exploration in reinforcement learning, as applied to Atari 2600 game-playing, has emphasized tackling difficult exploration prob… (voir plus)lems such as Montezuma's Revenge (Bellemare et al., 2016). Recently, bonus-based exploration methods, which explore by augmenting the environment reward, have reached above-human average performance on such domains. In this paper we reassess popular bonus-based exploration methods within a common evaluation framework. We combine Rainbow (Hessel et al., 2018) with different exploration bonuses and evaluate its performance on Montezuma's Revenge, Bellemare et al.'s set of hard of exploration games with sparse rewards, and the whole Atari 2600 suite. We find that while exploration bonuses lead to higher score on Montezuma's Revenge they do not provide meaningful gains over the simpler

2019-12-31

ICLR.cc/2020/Conference (poster)