Yoshua Bengio

jamal.abouhaibeh@mila.quebec

Biographie

*Pour toute demande média, veuillez écrire à medias@mila.quebec.

Pour plus d’information, contactez Cassidy MacNeil, adjointe principale et responsable des opérations cassidy.macneil@mila.quebec.

Reconnu comme une sommité mondiale en intelligence artificielle, Yoshua Bengio s’est surtout distingué par son rôle de pionnier en apprentissage profond, ce qui lui a valu le prix A. M. Turing 2018, le « prix Nobel de l’informatique », avec Geoffrey Hinton et Yann LeCun. Il est professeur titulaire à l’Université de Montréal, fondateur et conseiller scientifique de Mila – Institut québécois d’intelligence artificielle, et codirige en tant que senior fellow le programme Apprentissage automatique, apprentissage biologique de l'Institut canadien de recherches avancées (CIFAR). Il occupe également la fonction de conseiller spécial et directeur scientifique fondateur d’IVADO.

En 2018, il a été l’informaticien qui a recueilli le plus grand nombre de nouvelles citations au monde. En 2019, il s’est vu décerner le prestigieux prix Killam. Depuis 2022, il détient le plus grand facteur d’impact (h-index) en informatique à l’échelle mondiale. Il est fellow de la Royal Society de Londres et de la Société royale du Canada, et officier de l’Ordre du Canada.

Soucieux des répercussions sociales de l’IA et de l’objectif que l’IA bénéficie à tous, il a contribué activement à la Déclaration de Montréal pour un développement responsable de l’intelligence artificielle.

Étudiants actuels

Jamal Abou Haibeh

Collaborateur·rice alumni - McGill

Collaborateur·rice de recherche - Cambridge University

Superviseur⋅e principal⋅e :

David Rolnick

berkes.anais@mila.quebec

sanghyeok.choi@mila.quebec

Rim Assouel

Doctorat - UdeM

assouelr@mila.quebec

Xiaoyin Chen

Doctorat - UdeM

xiaoyin.chen@mila.quebec

Sanghyeok Choi

Collaborateur·rice de recherche - KAIST

Doctorat - UdeM

aniket.didolkar@mila.quebec

Doctorat - UdeM

Co-superviseur⋅e :

Guillaume Lajoie

eric.elmoznino@mila.quebec

Doctorat - UdeM

lena-nehale.ezzine@mila.quebec

Jean-Pierre Falet

Doctorat - UdeM

jean-pierre.falet@mila.quebec

Doctorat

pietro.greiner@mila.quebec

Mohsin Hasan

Doctorat - UdeM

mohsin.hasan@mila.quebec

thomas.jiralerspong@mila.quebec

Moksh Jain

Doctorat - UdeM

moksh.jain@mila.quebec

Doctorat - UdeM

Superviseur⋅e principal⋅e :

Guillaume Lajoie

Collaborateur·rice alumni - UdeM

younesse.kaddar@mila.quebec

Hyeonah Kim

Postdoctorat - UdeM

Superviseur⋅e principal⋅e :

Alex Hernández-García

hyeonah.kim@mila.quebec

Minsu Kim

Collaborateur·rice de recherche - UdeM

minsu.kim@mila.quebec

Collaborateur·rice de recherche - KOREA ADVANCED INSTITUTE OF SCIENCE AND TECHNOLOGY

jeongjae.lee@mila.quebec

cristian-dragos.manta@mila.quebec

Song LIU

Collaborateur·rice de recherche - s.o.

Collaborateur·rice de recherche

Cristian Dragos Manta

Doctorat - UdeM

Co-superviseur⋅e :

Dhanya Sridhar

Sarthak Mittal

Doctorat - UdeM

Superviseur⋅e principal⋅e :

Visiteur de recherche indépendant - UdeM

nguyenph@mila.quebec

Ali Parviz

Collaborateur·rice de recherche - Ying Wu Coll of Computing

ali.parviz@mila.quebec

Lena Podina

Collaborateur·rice de recherche - University of Waterloo

Superviseur⋅e principal⋅e :

David Rolnick

elena.podina@mila.quebec

jarrid.rector-brooks@mila.quebec

Jarrid Rector-Brooks

Doctorat - UdeM

Danyal REHMAN

Postdoctorat - UdeM

danyal.rehman@mila.quebec

oliver.richardson@mila.quebec

Oli RICHARDSON

Postdoctorat - UdeM

Camille Rochefort-Boulanger

Doctorat - UdeM

Superviseur⋅e principal⋅e :

Postdoctorat - UdeM

Co-superviseur⋅e :

Alex Hernández-García

divya.sharma@mila.quebec

Mélisande Astrid Crystal Teng

Collaborateur·rice alumni - UdeM

Co-superviseur⋅e :

Alex Tong

Collaborateur·rice alumni - UdeM

alexander.tong@mila.quebec

Collaborateur·rice de recherche - University of Cambridge

oszkar.urban@mila.quebec

donna.vakalis@mila.quebec

Donna Vakalis

Collaborateur·rice alumni - UdeM

Co-superviseur⋅e :

David Rolnick

Doctorat - UdeM

Superviseur⋅e principal⋅e :

Glen Berseth

siddarth.venkatraman@mila.quebec

Collaborateur·rice de recherche

joseph.viviano2@mila.quebec

Collaborateur·rice de recherche - UdeM

esmeralda.whitammer@mila.quebec

Collaborateur·rice alumni

omar.younis@mila.quebec

Collaborateur·rice de recherche - UdeM

taeyoung.yun@mila.quebec

Nicole Zhang

Doctorat - McGill

Superviseur⋅e principal⋅e :

Doctorat - UdeM

tianyu.zhang@mila.quebec

Skipper : combiner l’abstraction spatiale et temporelle afin d’améliorer la généralisation

Harry Zhao

Collaborateur·rice alumni - McGill

Superviseur⋅e principal⋅e :

Billets de blogue

Generic thumbnail for Mila Blog articles.

22 février 2024

par

Mingde Harry Zhao

Safa Alver

Harm van Seijen

Romain Laroche

Doina Precup

Yoshua Bengio

Mise à l’échelle au service du raisonnement et de l’apprentissage automatique basé sur un modèle

Scaling in the service of reasoning & model-based ML

4 avril 2023

par

Yoshua Bengio

Edward J. Hu

Une collaboration entre Mila et Relation Therapeutics pour découvrir in vitro de nouvelles associations médicamenteuses synergiques

A collaboration between Mila and Relation Therapeutics to discover novel synergistic combinations of drugs in vitro

23 mars 2022

par

Paul Bertin

Jake P. Taylor-King

Yoshua Bengio

Les réseaux de flot génératifs

15 mars 2022

par

Yoshua Bengio

Publications

Twin Networks: Using the Future as a Regularizer

Dmitriy Serdyuk

Nan Rosemary Ke

Alessandro Sordoni

Christopher Pal

Being able to model long-term dependencies in sequential data, such as text, has been among the long-standing challenges of recurrent neural… (voir plus) networks (RNNs). This issue is strictly related to the absence of explicit planning in current RNN architectures. More explicitly, the RNNs are trained to predict only the next token given previous ones. In this paper, we introduce a simple way of encouraging the RNNs to plan for the future. In order to accomplish this, we introduce an additional neural network which is trained to generate the sequence in reverse order, and we require closeness between the states of the forward RNN and backward RNN that predict the same token. At each step, the states of the forward RNN are required to match the future information contained in the backward states. We hypothesize that the approach eases modeling of long-term dependencies thus helping in generating more globally consistent samples. The model trained with conditional generation for a speech recognition task achieved 12\% relative improvement (CER of 6.7 compared to a baseline of 7.6).

2017-08-21

ArXiv (prépublication)

Dynamic Layer Normalization for Adaptive Neural Acoustic Modeling in Speech Recognition

Taesup Kim

Inchul Song

Layer normalization is a recently introduced technique for normalizing the activities of neurons in deep neural networks to improve the trai… (voir plus)ning speed and stability. In this paper, we introduce a new layer normalization technique called Dynamic Layer Normalization (DLN) for adaptive neural acoustic modeling in speech recognition. By dynamically generating the scaling and shifting parameters in layer normalization, DLN adapts neural acoustic models to the acoustic variability arising from various factors such as speakers, channel noises, and environments. Unlike other adaptive acoustic models, our proposed approach does not require additional adaptation data or speaker information such as i-vectors. Moreover, the model size is fixed as it dynamically generates adaptation parameters. We apply our proposed DLN to deep bidirectional LSTM acoustic models and evaluate them on two benchmark datasets for large vocabulary ASR experiments: WSJ and TED-LIUM release 2. The experimental results show that our DLN improves neural acoustic models in terms of transcription accuracy by dynamically adapting to various speakers and environments.

2017-08-19

Interspeech 2017 (publié)

Improving Speech Recognition by Revising Gated Recurrent Units

Mirco Ravanaelli

Philemon Brakel

Maurizio Omologo

Speech recognition is largely taking advantage of deep learning, showing that substantial benefits can be obtained by modern Recurrent Neura… (voir plus)l Networks (RNNs). The most popular RNNs are Long Short-Term Memory (LSTMs), which typically reach state-of-the-art performance in many tasks thanks to their ability to learn long-term dependencies and robustness to vanishing gradients. Nevertheless, LSTMs have a rather complex design with three multiplicative gates, that might impair their efficient implementation. An attempt to simplify LSTMs has recently led to Gated Recurrent Units (GRUs), which are based on just two multiplicative gates. This paper builds on these efforts by further revising GRUs and proposing a simplified architecture potentially more suitable for speech recognition. The contribution of this work is two-fold. First, we suggest to remove the reset gate in the GRU design, resulting in a more efficient single-gate architecture. Second, we propose to replace tanh with ReLU activations in the state update equations. Results show that, in our implementation, the revised architecture reduces the per-epoch training time with more than 30% and consistently improves recognition performance across different tasks, input features, and noisy conditions when compared to a standard GRU.

2017-08-19

Interspeech 2017 (publié)

A Closer Look at Memorization in Deep Networks

Devansh Arpit

Stanisław Jastrzębski

David Krueger

Maxinder S. Kanwal

We examine the role of memorization in deep learning, drawing connections to capacity, generalization, and adversarial robustness. While dee… (voir plus)p networks are capable of memorizing noise data, our results suggest that they tend to prioritize learning simple patterns first. In our experiments, we expose qualitative differences in gradient-based optimization of deep neural networks (DNNs) on noise vs. real data. We also demonstrate that for appropriately tuned explicit regularization (e.g., dropout) we can degrade DNN training performance on noise datasets without compromising generalization on real data. Our analysis suggests that the notions of effective capacity which are dataset independent are unlikely to explain the generalization performance of deep networks when trained with gradient based methods because training data itself plays an important role in determining the degree of memorization.

2017-08-05

International Conference on Machine Learning (inconnu)

proceedings.mlr.press

Plan, Attend, Generate: Character-Level Neural Machine Translation with Planning

Caglar Gulçehre

Francis Dutil

Adam Trischler

We investigate the integration of a planning mechanism into an encoder-decoder architecture with attention. We develop a model that can plan… (voir plus) ahead when it computes alignments between the source and target sequences not only for a single time-step but for the next k time-steps as well by constructing a matrix of proposed future alignments and a commitment vector that governs whether to follow or recompute the plan. This mechanism is inspired by strategic attentive reader and writer (STRAW) model, a recent neural architecture for planning with hierarchical reinforcement learning that can also learn higher level temporal abstractions. Our proposed model is end-to-end trainable with differentiable operations. We show that our model outperforms strong baselines on character-level translation task from WMT’15 with fewer parameters and computes alignments that are qualitatively intuitive.

2017-07-31

Proceedings of the 2nd Workshop on Representation Learning for NLP (publié)

Multiscale sequence modeling with a learned dictionary

We propose a generalization of neural network sequence models. Instead of predicting one symbol at a time, our multi-scale model makes predi… (voir plus)ctions over multiple, potentially overlapping multi-symbol tokens. A variation of the byte-pair encoding (BPE) compression algorithm is used to learn the dictionary of tokens that the model is trained with. When applied to language modelling, our model has the flexibility of character-level models while maintaining many of the performance benefits of word-level models. Our experiments show that this model performs better than a regular LSTM on language modeling tasks, especially for smaller models.

2017-07-02

ArXiv (prépublication)

Variance Regularizing Adversarial Learning

Karan Grewal

R Devon Hjelm

We study how, in generative adversarial networks, variance in the discriminator's output affects the generator's ability to learn the data d… (voir plus)istribution. In particular, we contrast the results from various well-known techniques for training GANs when the discriminator is near-optimal and updated multiple times per update to the generator. As an alternative, we propose an additional method to train GANs by explicitly modeling the discriminator's output as a bi-modal Gaussian distribution over the real/fake indicator variables. In order to do this, we train the Gaussian classifier to match the target bi-modal distribution implicitly through meta-adversarial training. We observe that our new method, when trained together with a strong discriminator, provides meaningful, non-vanishing gradients.

2017-07-01

ArXiv (prépublication)

openreview.net

Towards an Automatic Turing Test: Learning to Evaluate Dialogue Responses

Ryan Lowe

Michael Noseworthy

Iulian V. Serban

Nicolas A.-Gontier

Joelle Pineau

Automatically evaluating the quality of dialogue responses for unstructured domains is a challenging problem. Unfortunately, existing automa… (voir plus)tic evaluation metrics are biased and correlate very poorly with human judgements of response quality. Yet having an accurate automatic evaluation procedure is crucial for dialogue research, as it allows rapid prototyping and testing of new models with fewer expensive human evaluations. In response to this challenge, we formulate automatic dialogue evaluation as a learning problem. We present an evaluation model (ADEM) that learns to predict human-like scores to input responses, using a new dataset of human response scores. We show that the ADEM model's predictions correlate significantly, and at a level much higher than word-overlap metrics such as BLEU, with human judgements at both the utterance and system-level. We also show that ADEM can generalize to evaluating dialogue models unseen during training, an important step for automatic dialogue evaluation.

2017-06-30

Annual Meeting of the Association for Computational Linguistics (publié)

Towards more hardware-friendly deep learning

2017-06-23

TiML '17 (publié)

Deep Learning for Patient-Specific Kidney Graft Survival Analysis

Margaux Luck

Tristan Sylvain

Heloise Cardinal

Andrea Lodi

An accurate model of patient-specific kidney graft survival distributions can help to improve shared-decision making in the treatment and ca… (voir plus)re of patients. In this paper, we propose a deep learning method that directly models the survival function instead of estimating the hazard function to predict survival times for graft patients based on the principle of multi-task learning. By learning to jointly predict the time of the event, and its rank in the cox partial log likelihood framework, our deep learning approach outperforms, in terms of survival time prediction quality and concordance index, other common methods for survival analysis, including the Cox Proportional Hazards model and a network trained on the cox partial log-likelihood.

2017-05-28

ArXiv (prépublication)

Char2Wav: End-to-End Speech Synthesis

Jose Sotelo

2017-02-16

International Conference on Learning Representations (inconnu)

openreview.net

A Hierarchical Latent Variable Encoder-Decoder Model for Generating Dialogues

Iulian V. Serban

Sequential data often possesses a hierarchical structure with complex dependencies between subsequences, such as found between the utterance… (voir plus)s in a dialogue. In an effort to model this kind of generative process, we propose a neural network-based generative architecture, with latent stochastic variables that span a variable number of time steps. We apply the proposed model to the task of dialogue response generation and compare it with recent neural network architectures. We evaluate the model performance through automatic evaluation metrics and by carrying out a human evaluation. The experiments demonstrate that our model improves upon recently proposed models and that the latent variables facilitate the generation of long outputs and maintain the context.

2017-02-11

Proceedings of the AAAI Conference on Artificial Intelligence (publié)