Yoshua Bengio

jamal.abouhaibeh@mila.quebec

Biographie

*Pour toute demande média, veuillez écrire à medias@mila.quebec.

Pour plus d’information, contactez Cassidy MacNeil, adjointe principale et responsable des opérations cassidy.macneil@mila.quebec.

Reconnu comme une sommité mondiale en intelligence artificielle, Yoshua Bengio s’est surtout distingué par son rôle de pionnier en apprentissage profond, ce qui lui a valu le prix A. M. Turing 2018, le « prix Nobel de l’informatique », avec Geoffrey Hinton et Yann LeCun. Il est professeur titulaire à l’Université de Montréal, fondateur et conseiller scientifique de Mila – Institut québécois d’intelligence artificielle, et codirige en tant que senior fellow le programme Apprentissage automatique, apprentissage biologique de l'Institut canadien de recherches avancées (CIFAR). Il occupe également la fonction de conseiller spécial et directeur scientifique fondateur d’IVADO.

En 2018, il a été l’informaticien qui a recueilli le plus grand nombre de nouvelles citations au monde. En 2019, il s’est vu décerner le prestigieux prix Killam. Depuis 2022, il détient le plus grand facteur d’impact (h-index) en informatique à l’échelle mondiale. Il est fellow de la Royal Society de Londres et de la Société royale du Canada, et officier de l’Ordre du Canada.

Soucieux des répercussions sociales de l’IA et de l’objectif que l’IA bénéficie à tous, il a contribué activement à la Déclaration de Montréal pour un développement responsable de l’intelligence artificielle.

Étudiants actuels

Jamal Abou Haibeh

Collaborateur·rice alumni - McGill

Collaborateur·rice de recherche - Cambridge University

Superviseur⋅e principal⋅e :

David Rolnick

berkes.anais@mila.quebec

sanghyeok.choi@mila.quebec

Rim Assouel

Doctorat - UdeM

assouelr@mila.quebec

Xiaoyin Chen

Doctorat - UdeM

xiaoyin.chen@mila.quebec

Sanghyeok Choi

Collaborateur·rice de recherche - KAIST

Doctorat - UdeM

aniket.didolkar@mila.quebec

Doctorat - UdeM

Co-superviseur⋅e :

Guillaume Lajoie

eric.elmoznino@mila.quebec

Doctorat - UdeM

lena-nehale.ezzine@mila.quebec

Jean-Pierre Falet

Doctorat - UdeM

jean-pierre.falet@mila.quebec

Doctorat

pietro.greiner@mila.quebec

Mohsin Hasan

Doctorat - UdeM

mohsin.hasan@mila.quebec

thomas.jiralerspong@mila.quebec

Moksh Jain

Doctorat - UdeM

moksh.jain@mila.quebec

Doctorat - UdeM

Superviseur⋅e principal⋅e :

Guillaume Lajoie

Collaborateur·rice alumni - UdeM

younesse.kaddar@mila.quebec

Minsu Kim

Collaborateur·rice de recherche - UdeM

minsu.kim@mila.quebec

Hyeonah Kim

Postdoctorat - UdeM

Superviseur⋅e principal⋅e :

Alex Hernández-García

hyeonah.kim@mila.quebec

Jeongjae Lee

Collaborateur·rice de recherche - KOREA ADVANCED INSTITUTE OF SCIENCE AND TECHNOLOGY

jeongjae.lee@mila.quebec

cristian-dragos.manta@mila.quebec

Song LIU

Collaborateur·rice de recherche - s.o.

Collaborateur·rice de recherche

Cristian Dragos Manta

Doctorat - UdeM

Co-superviseur⋅e :

Dhanya Sridhar

Sarthak Mittal

Doctorat - UdeM

Superviseur⋅e principal⋅e :

Visiteur de recherche indépendant - UdeM

nguyenph@mila.quebec

Ali Parviz

Collaborateur·rice de recherche - Ying Wu Coll of Computing

ali.parviz@mila.quebec

Lena Podina

Collaborateur·rice de recherche - University of Waterloo

Superviseur⋅e principal⋅e :

David Rolnick

elena.podina@mila.quebec

jarrid.rector-brooks@mila.quebec

Jarrid Rector-Brooks

Doctorat - UdeM

Danyal REHMAN

Postdoctorat - UdeM

danyal.rehman@mila.quebec

oliver.richardson@mila.quebec

Oli RICHARDSON

Postdoctorat - UdeM

Camille Rochefort-Boulanger

Doctorat - UdeM

Superviseur⋅e principal⋅e :

Postdoctorat - UdeM

Co-superviseur⋅e :

Alex Hernández-García

divya.sharma@mila.quebec

Mélisande Astrid Crystal Teng

Collaborateur·rice alumni - UdeM

Co-superviseur⋅e :

Alex Tong

Collaborateur·rice alumni - UdeM

alexander.tong@mila.quebec

Collaborateur·rice de recherche - University of Cambridge

oszkar.urban@mila.quebec

donna.vakalis@mila.quebec

Donna Vakalis

Collaborateur·rice alumni - UdeM

Co-superviseur⋅e :

David Rolnick

Doctorat - UdeM

Superviseur⋅e principal⋅e :

Glen Berseth

siddarth.venkatraman@mila.quebec

Collaborateur·rice de recherche

joseph.viviano2@mila.quebec

Collaborateur·rice de recherche - UdeM

esmeralda.whitammer@mila.quebec

Collaborateur·rice alumni

omar.younis@mila.quebec

Collaborateur·rice de recherche - UdeM

taeyoung.yun@mila.quebec

Skipper : combiner l’abstraction spatiale et temporelle afin d’améliorer la généralisation

Tianyu Zhang

Doctorat - UdeM

tianyu.zhang@mila.quebec

Doctorat - McGill

Superviseur⋅e principal⋅e :

Harry Zhao

Collaborateur·rice alumni - McGill

Superviseur⋅e principal⋅e :

Billets de blogue

Generic thumbnail for Mila Blog articles.

22 février 2024

par

Mingde Harry Zhao

Safa Alver

Harm van Seijen

Romain Laroche

Doina Precup

Yoshua Bengio

Mise à l’échelle au service du raisonnement et de l’apprentissage automatique basé sur un modèle

Scaling in the service of reasoning & model-based ML

4 avril 2023

par

Yoshua Bengio

Edward J. Hu

Une collaboration entre Mila et Relation Therapeutics pour découvrir in vitro de nouvelles associations médicamenteuses synergiques

A collaboration between Mila and Relation Therapeutics to discover novel synergistic combinations of drugs in vitro

23 mars 2022

par

Paul Bertin

Jake P. Taylor-King

Yoshua Bengio

Les réseaux de flot génératifs

15 mars 2022

par

Yoshua Bengio

Publications

FigureQA: An Annotated Figure Dataset for Visual Reasoning

Samira Ebrahimi Kahou

Adam Atkinson

Vincent Michalski

Ákos Kádár

Adam Trischler

We introduce FigureQA, a visual reasoning corpus of over one million question-answer pairs grounded in over 100,000 images. The images are s… (voir plus)ynthetic, scientific-style figures from five classes: line plots, dot-line plots, vertical and horizontal bar graphs, and pie charts. We formulate our reasoning task by generating questions from 15 templates; questions concern various relationships between plot elements and examine characteristics like the maximum, the minimum, area-under-the-curve, smoothness, and intersection. To resolve, such questions often require reference to multiple plot elements and synthesis of information distributed spatially throughout a figure. To facilitate the training of machine learning systems, the corpus also includes side data that can be used to formulate auxiliary objectives. In particular, we provide the numerical data used to generate each figure as well as bounding-box annotations for all plot elements. We study the proposed visual reasoning task by training several models, including the recently proposed Relation Network as a strong baseline. Preliminary results indicate that the task poses a significant machine learning challenge. We envision FigureQA as a first step towards developing models that can intuitively recognize patterns from visual representations of data.

2017-12-31

ICLR.cc/2018/Conference (Invite to Workshop Track)

Fraternal Dropout

Recurrent neural networks (RNNs) are important class of architectures among neural networks useful for language modeling and sequential pred… (voir plus)iction. However, optimizing RNNs is known to be harder compared to feed-forward neural networks. A number of techniques have been proposed in literature to address this problem. In this paper we propose a simple technique called fraternal dropout that takes advantage of dropout to achieve this goal. Specifically, we propose to train two identical copies of an RNN (that share parameters) with different dropout masks while minimizing the difference between their (pre-softmax) predictions. In this way our regularization encourages the representations of RNNs to be invariant to dropout mask, thus being robust. We show that our regularization term is upper bounded by the expectation-linear dropout objective which has been shown to address the gap due to the difference between the train and inference phases of dropout. We evaluate our model and achieve state-of-the-art results in sequence modeling tasks on two benchmark datasets - Penn Treebank and Wikitext-2. We also show that our approach leads to performance improvement by a significant margin in image captioning (Microsoft COCO) and semi-supervised (CIFAR-10) tasks.

2017-12-31

ICLR.cc/2018/Conference (poster)

Graph Attention Networks

Petar Veličković

Guillem Cucurull

Arantxa Casanova

Adriana Romero

Pietro Lio

We present graph attention networks (GATs), novel neural network architectures that operate on graph-structured data, leveraging masked self… (voir plus)-attentional layers to address the shortcomings of prior methods based on graph convolutions or their approximations. By stacking layers in which nodes are able to attend over their neighborhoods' features, we enable (implicitly) specifying different weights to different nodes in a neighborhood, without requiring any kind of costly matrix operation (such as inversion) or depending on knowing the graph structure upfront. In this way, we address several key challenges of spectral-based graph neural networks simultaneously, and make our model readily applicable to inductive as well as transductive problems. Our GAT models have achieved or matched state-of-the-art results across four established transductive and inductive graph benchmarks: the Cora, Citeseer and Pubmed citation network datasets, as well as a protein-protein interaction dataset (wherein test graphs remain unseen during training).

2017-12-31

ICLR.cc/2018/Conference (poster)

Image-to-image translation for cross-domain disentanglement

Abel Gonzalez-Garcia

Joost van de Weijer

Deep image translation methods have recently shown excellent results, outputting high-quality images covering multiple modes of the data dis… (voir plus)tribution. There has also been increased interest in disentangling the internal representations learned by deep methods to further improve their performance and achieve a finer control. In this paper, we bridge these two objectives and introduce the concept of cross-domain disentanglement. We aim to separate the internal representation into three parts. The shared part contains information for both domains. The exclusive parts, on the other hand, contain only factors of variation that are particular to each domain. We achieve this through bidirectional image translation based on Generative Adversarial Networks and cross-domain autoencoders, a novel network component. Our model offers multiple advantages. We can output diverse samples covering multiple modes of the distributions of both domains, perform domain-specific image transfer and interpolation, and cross-domain retrieval without the need of labeled data, only paired images. We compare our model to the state-of-the-art in multi-modal image translation and achieve better results for translation on challenging datasets as well as for cross-domain retrieval on realistic datasets.

2017-12-31

NeurIPS (publié)

LATTER M INIMA WITH SGD

Stanisław Jastrzębski

Amos Storkey

2017-12-31

(publié)

www.semanticscholar.org

Learning General Purpose Distributed Sentence Representations via Large Scale Multi-task Learning

Sandeep Subramanian

Adam Trischler

Christopher Pal

A lot of the recent success in natural language processing (NLP) has been driven by distributed vector representations of words trained on l… (voir plus)arge amounts of text in an unsupervised manner. These representations are typically used as general purpose features for words across a range of NLP problems. However, extending this success to learning representations of sequences of words, such as sentences, remains an open problem. Recent work has explored unsupervised as well as supervised learning techniques with different training objectives to learn general purpose fixed-length sentence representations. In this work, we present a simple, effective multi-task learning framework for sentence representations that combines the inductive biases of diverse training objectives in a single model. We train this model on several data sources with multiple training objectives on over 100 million sentences. Extensive experiments demonstrate that sharing a single recurrent sentence encoder across weakly related tasks leads to consistent improvements over previous methods. We present substantial improvements in the context of transfer learning and low-resource settings using our learned general-purpose representations.

2017-12-31

ICLR (Poster) (publié)

Residual Connections Encourage Iterative Inference

Stanisław Jastrzębski

Tong Che

Residual networks (Resnets) have become a prominent architecture in deep learning. However, a comprehensive understanding of Resnets is stil… (voir plus)l a topic of ongoing research. A recent view argues that Resnets perform iterative refinement of features. We attempt to further expose properties of this aspect. To this end, we study Resnets both analytically and empirically. We formalize the notion of iterative refinement in Resnets by showing that residual connections naturally encourage features of residual blocks to move along the negative gradient of loss as we go from one block to the next. In addition, our empirical analysis suggests that Resnets are able to perform both representation learning and iterative refinement. In general, a Resnet block tends to concentrate representation learning behavior in the first few layers while higher layers perform iterative refinement of features. Finally we observe that sharing residual layers naively leads to representation explosion and counterintuitively, overfitting, and we show that simple existing strategies can help alleviating this problem.

2017-12-31

International Conference on Learning Representations (poster)

Sparse Attentive Backtracking: Temporal Credit Assignment Through Reminding

Nan Rosemary Ke

Michael C. Mozer

Learning long-term dependencies in extended temporal sequences requires credit assignment to events far back in the past. The most common me… (voir plus)thod for training recurrent neural networks, back-propagation through time (BPTT), requires credit information to be propagated backwards through every single step of the forward computation, potentially over thousands or millions of time steps. This becomes computationally expensive or even infeasible when used with long sequences. Importantly, biological brains are unlikely to perform such detailed reverse replay over very long sequences of internal states (consider days, months, or years.) However, humans are often reminded of past memories or mental states which are associated with the current mental state. We consider the hypothesis that such memory associations between past and present could be used for credit assignment through arbitrarily long sequences, propagating the credit assigned to the current state to the associated past state. Based on this principle, we study a novel algorithm which only back-propagates through a few of these temporal skip connections, realized by a learned attention mechanism that associates current states with relevant past states. We demonstrate in experiments that our method matches or outperforms regular BPTT and truncated BPTT in tasks involving particularly long-term dependencies, but without requiring the biologically implausible backward replay through the whole history of states. Additionally, we demonstrate that the proposed method transfers to longer sequences significantly better than LSTMs trained with BPTT and LSTMs trained with full self-attention.

2017-12-31

Neural Information Processing Systems (publié)

Twin Networks: Matching the Future for Sequence Generation

Dmitriy Serdyuk

Nan Rosemary Ke

Alessandro Sordoni

Adam Trischler

Christopher Pal

We propose a simple technique for encouraging generative RNNs to plan ahead. We train a "backward" recurrent network to generate a given seq… (voir plus)uence in reverse order, and we encourage states of the forward model to predict cotemporal states of the backward model. The backward network is used only during training, and plays no role during sampling or inference. We hypothesize that our approach eases modeling of long-term dependencies by implicitly forcing the forward states to hold information about the longer-term future (as contained in the backward states). We show empirically that our approach achieves 9% relative improvement for a speech recognition task, and achieves significant improvement on a COCO caption generation task.

2017-12-31

ICLR (Poster) (publié)

Dendritic error backpropagation in deep cortical microcircuits

João Sacramento

Rui Ponte Costa

Walter Senn

Animal behaviour depends on learning to associate sensory stimuli with the desired motor command. Understanding how the brain orchestrates t… (voir plus)he necessary synaptic modifications across different brain areas has remained a longstanding puzzle. Here, we introduce a multi-area neuronal network model in which synaptic plasticity continuously adapts the network towards a global desired output. In this model synaptic learning is driven by a local dendritic prediction error that arises from a failure to predict the top-down input given the bottom-up activities. Such errors occur at apical dendrites of pyramidal neurons where both long-range excitatory feedback and local inhibitory predictions are integrated. When local inhibition fails to match excitatory feedback an error occurs which triggers plasticity at bottom-up synapses at basal dendrites of the same pyramidal neurons. We demonstrate the learning capabilities of the model in a number of tasks and show that it approximates the classical error backpropagation algorithm. Finally, complementing this cortical circuit with a disinhibitory mechanism enables attention-like stimulus denoising and generation. Our framework makes several experimental predictions on the function of dendritic integration and cortical microcircuits, is consistent with recent observations of cross-area learning, and suggests a biological implementation of deep learning.

2017-12-29

ArXiv (prépublication)

ObamaNet: Photo-realistic lip-sync from text

Rithesh Kumar

Jose Sotelo

Kundan Kumar

Alexandre De Brébisson

We present ObamaNet, the first architecture that generates both audio and synchronized photo-realistic lip-sync videos from any new text. Co… (voir plus)ntrary to other published lip-sync approaches, ours is only composed of fully trainable neural modules and does not rely on any traditional computer graphics methods. More precisely, we use three main modules: a text-to-speech network based on Char2Wav, a time-delayed LSTM to generate mouth-keypoints synced to the audio, and a network based on Pix2Pix to generate the video frames conditioned on the keypoints.

2017-12-05

arXiv (prépublication)

Use machine learning to find energy materials.

Phil De Luna

Jennifer N. Wei

Al'an Aspuru-guzik

E. Sargent

2017-12-05

Nature (publié)