Yoshua Bengio

jamal.abouhaibeh@mila.quebec

Biography

*For media requests, please write to medias@mila.quebec.

For more information please contact Cassidy MacNeil, Senior Assistant and Operation Lead at cassidy.macneil@mila.quebec.

Yoshua Bengio is recognized worldwide as a leading expert in AI. He is most known for his pioneering work in deep learning, which earned him the 2018 A.M. Turing Award, “the Nobel Prize of computing,” with Geoffrey Hinton and Yann LeCun.

Bengio is a full professor at Université de Montréal, and the founder and scientific advisor of Mila – Quebec Artificial Intelligence Institute. He is also a senior fellow at CIFAR and co-directs its Learning in Machines & Brains program, serves as special advisor and founding scientific director of IVADO, and holds a Canada CIFAR AI Chair.

In 2019, Bengio was awarded the prestigious Killam Prize and in 2022, he was the most cited computer scientist in the world by h-index. He is a Fellow of the Royal Society of London, Fellow of the Royal Society of Canada, Knight of the Legion of Honor of France and Officer of the Order of Canada. In 2023, he was appointed to the UN’s Scientific Advisory Board for Independent Advice on Breakthroughs in Science and Technology.

Concerned about the social impact of AI, Bengio helped draft the Montréal Declaration for the Responsible Development of Artificial Intelligence and continues to raise awareness about the importance of mitigating the potentially catastrophic risks associated with future AI systems.

Current Students

Jamal Abou Haibeh

Collaborating Alumni - McGill University

Berkes Anaïs

Collaborating researcher - Cambridge University

Principal supervisor :

berkes.anais@mila.quebec

Rim Assouel

PhD - Université de Montréal

assouelr@mila.quebec

Shahana Chatterjee

Collaborating researcher - N/A

Principal supervisor :

shahana.chatterjee@mila.quebec

sanghyeok.choi@mila.quebec

Xiaoyin Chen

PhD - Université de Montréal

xiaoyin.chen@mila.quebec

Sanghyeok Choi

Collaborating researcher - KAIST

PhD - Université de Montréal

aniket.didolkar@mila.quebec

PhD - Université de Montréal

Co-supervisor :

Guillaume Lajoie

eric.elmoznino@mila.quebec

PhD - Université de Montréal

lena-nehale.ezzine@mila.quebec

Jean-Pierre Falet

PhD - Université de Montréal

jean-pierre.falet@mila.quebec

PhD

pietro.greiner@mila.quebec

Mohsin Hasan

PhD - Université de Montréal

mohsin.hasan@mila.quebec

thomas.jiralerspong@mila.quebec

Moksh Jain

PhD - Université de Montréal

moksh.jain@mila.quebec

PhD - Université de Montréal

Principal supervisor :

Guillaume Lajoie

Collaborating Alumni - Université de Montréal

younesse.kaddar@mila.quebec

Hyeonah Kim

Postdoctorate - Université de Montréal

Principal supervisor :

Alex Hernández-García

hyeonah.kim@mila.quebec

Minsu Kim

Collaborating researcher - Université de Montréal

minsu.kim@mila.quebec

Research Intern - KOREA ADVANCED INSTITUTE OF SCIENCE AND TECHNOLOGY

jeongjae.lee@mila.quebec

cristian-dragos.manta@mila.quebec

Song LIU

Collaborating researcher - s.o.

Collaborating researcher

Cristian Dragos Manta

PhD - Université de Montréal

Co-supervisor :

Dhanya Sridhar

Sarthak Mittal

PhD - Université de Montréal

Principal supervisor :

Independent visiting researcher - Université de Montréal

nguyenph@mila.quebec

Ali Parviz

Collaborating researcher - Ying Wu Coll of Computing

ali.parviz@mila.quebec

Lena Podina

Collaborating researcher - University of Waterloo

Principal supervisor :

elena.podina@mila.quebec

jarrid.rector-brooks@mila.quebec

Jarrid Rector-Brooks

PhD - Université de Montréal

Danyal REHMAN

Postdoctorate - Université de Montréal

danyal.rehman@mila.quebec

oliver.richardson@mila.quebec

Oli RICHARDSON

Postdoctorate - Université de Montréal

Camille Rochefort-Boulanger

PhD - Université de Montréal

Principal supervisor :

Postdoctorate - Université de Montréal

Co-supervisor :

Alex Hernández-García

divya.sharma@mila.quebec

Mélisande Astrid Crystal Teng

Collaborating Alumni - Université de Montréal

Co-supervisor :

Alex Tong

Collaborating Alumni - Université de Montréal

alexander.tong@mila.quebec

Collaborating researcher - University of Cambridge

oszkar.urban@mila.quebec

Donna Vakalis

Collaborating Alumni - Université de Montréal

Co-supervisor :

donna.vakalis@mila.quebec

PhD - Université de Montréal

Principal supervisor :

Glen Berseth

siddarth.venkatraman@mila.quebec

Collaborating researcher - Université de Montréal

esmeralda.whitammer@mila.quebec

Collaborating Alumni

omar.younis@mila.quebec

Collaborating researcher - Université de Montréal

taeyoung.yun@mila.quebec

Skipper: Combining Spatial and Temporal Abstraction for Better Generalization

Tianyu Zhang

PhD - Université de Montréal

tianyu.zhang@mila.quebec

PhD - McGill University

Principal supervisor :

Blog Posts

Generic thumbnail for Mila Blog articles.

February 22, 2024

Mingde Harry Zhao

Safa Alver

Harm van Seijen

Romain Laroche

Doina Precup

Yoshua Bengio

Scaling in the Service of Reasoning & Model-Based ML

April 4, 2023

Yoshua Bengio

Edward J. Hu

A collaboration between Mila and Relation Therapeutics to discover novel synergistic combinations of drugs in vitro

March 23, 2022

Paul Bertin

Jake P. Taylor-King

Yoshua Bengio

March 15, 2022

Generative Flow Networks

Yoshua Bengio

Publications

Modeling Cloud Reflectance Fields using Conditional Generative Adversarial Networks

Mustafa Alghali

We introduce a conditional Generative Adversarial Network (cGAN) approach to generate cloud reflectance fields (CRFs) conditioned on large s… (see more)cale meteorological variables such as sea surface temperature and relative humidity. We show that our trained model can generate realistic CRFs from the corresponding meteorological observations, which represents a step towards a data-driven framework for stochastic cloud parameterization.

2020-02-09

arXiv (preprint)

Using Simulated Data to Generate Images of Climate Change

Gautier Cosne

Adrien Juraver

Mélisande Teng

Victor Schmidt

Vahe Vardanyan

Alexandra Luccioni

Generative adversarial networks (GANs) used in domain adaptation tasks have the ability to generate images that are both realistic and perso… (see more)nalized, transforming an input image while maintaining its identifiable characteristics. However, they often require a large quantity of training data to produce high-quality images in a robust way, which limits their usability in cases when access to data is limited. In our paper, we explore the potential of using images from a simulated 3D environment to improve a domain adaptation task carried out by the MUNIT architecture, aiming to use the resulting images to raise awareness of the potential future impacts of climate change.

2020-01-25

ArXiv (preprint)

COVI White Paper - Version 1.1

Hannah Alsdurf

Edmond Belliveau

Tristan Deleu

Prateek Gupta

Daphne Ippolito

Richard Janda

Max Jarvie

Tyler Kolody

Sekoul Krastev

Tegan Maharaj

Robert Obryk

Dan Pilat

Valérie Pisano

Benjamin Prud’homme

Meng Qu

Nasim Rahaman

Irina Rish

Jean-François Rousseau

Victor Schmidt … (see 5 more)

Abhinav Sharma

Brooke Struck

Jian Tang

Martin Weiss

Yun William Yu

The SARS-CoV-2 (Covid-19) pandemic has caused significant strain on public health institutions around the world. Contact tracing is an essen… (see more)tial tool to change the course of the Covid-19 pandemic. Manual contact tracing of Covid-19 cases has significant challenges that limit the ability of public health authorities to minimize community infections. Personalized peer-to-peer contact tracing through the use of mobile apps has the potential to shift the paradigm. Some countries have deployed centralized tracking systems, but more privacy-protecting decentralized systems offer much of the same benefit without concentrating data in the hands of a state authority or for-profit corporations. Machine learning methods can circumvent some of the limitations of standard digital tracing by incorporating many clues and their uncertainty into a more graded and precise estimation of infection risk. The estimated risk can provide early risk awareness, personalized recommendations and relevant information to the user. Finally, non-identifying risk data can inform epidemiological models trained jointly with the machine learning predictor. These models can provide statistical evidence for the importance of factors involved in disease transmission. They can also be used to monitor, evaluate and optimize health policy and (de)confinement scenarios according to medical and economic productivity indicators. However, such a strategy based on mobile apps and machine learning should proactively mitigate potential ethical and privacy risks, which could have substantial impacts on society (not only impacts on health but also impacts such as stigmatization and abuse of personal data). Here, we present an overview of the rationale, design, ethical considerations and privacy strategy of `COVI,' a Covid-19 public peer-to-peer contact tracing and risk awareness mobile application developed in Canada.

2019-12-31

International Conference on Machine Learning (published)

GraphMix: Improved Training of Graph Neural Networks for Semi-Supervised Learning

Juho Kannala

We present GraphMix , a regularized training scheme for Graph Neural Network based semi-supervised object classiﬁcation, leveraging the re… (see more)cent advances in the regularization of classical deep neural networks. Speciﬁcally, we pro-pose a uniﬁed approach in which we train a fully-connected network jointly with the graph neural network via parameter sharing, interpolation-based regularization and self-predicted-targets. Our proposed method is architecture agnostic in the sense that it can be applied to any variant of graph neural networks which applies a parametric transformation to the features of the graph nodes. Despite its simplicity, with GraphMix we can consistently improve results and achieve or closely match state-of-the-art performance using even simpler architectures such as Graph Convolutional Networks, across three established graph benchmarks: Cora, Citeseer and Pubmed citation network datasets, as well as three newly proposed datasets :Cora-Full, Co-author-CS and Co-author-Physics.

2019-12-31

(published)

Joint Learning of Generative Translator and Classifier for Visually Similar Classes

Byungin Yoo

Tristan Sylvain

Junmo Kim

In this paper, we propose a Generative Translation Classification Network (GTCN) for improving visual classification accuracy in settings wh… (see more)ere classes are visually similar and data is scarce. For this purpose, we propose joint learning from a scratch to train a classifier and a generative stochastic translation network end-to-end. The translation network is used to perform on-line data augmentation across classes, whereas previous works have mostly involved domain adaptation. To help the model further benefit from this data-augmentation, we introduce an adaptive fade-in loss and a quadruplet loss. We perform experiments on multiple datasets to demonstrate the proposed method’s performance in varied settings. Of particular interest, training on 40% of the dataset is enough for our model to surpass the performance of baselines trained on the full dataset. When our architecture is trained on the full dataset, we achieve comparable performance with state-of-the-art methods despite using a light-weight architecture.

2019-12-31

IEEE Access (published)

Learning Classical Planning Transition Functions by Deep Neural Networks

Michaela Urbanovská

Ian G Goodfellow

Aaron Courville

2019-12-31

(published)

Learning Long-term Dependencies Using Cognitive Inductive Biases in Self-attention RNNs

Attention and self-attention mechanisms, inspired by cognitive processes, are now central to state-of-the-art deep learning on sequential ta… (see more)sks. However, most recent progress hinges on heuristic approaches that rely on considerable memory and computational resources that scale poorly. In this work, we propose a relevancy screening mechanism, inspired by the cognitive process of memory consolidation, that allows for a scalable use of sparse self-attention with recurrence. We use simple numerical experiments to demonstrate that this mechanism helps enable recurrent systems on generalization and transfer learning tasks. Based on our results, we propose a concrete direction of research to improve scalability and generalization of attentive recurrent networks.

2019-12-31

(published)

Learning the Arrow of Time for Problems in Reinforcement Learning

Nasim Rahaman

Steffen Wolf

Anirudh Goyal

Roman Remme

2019-12-31

ICLR (published)

Leveraging a Fully Differentiable Integrated Assessment Model for RL and Inference

Koen Ponse

Kai-Hendrik Cohrs

Phillip Wozny

Andrew Robert Williams

Tianyu Zhang

Erman Acar

Aske Plaat

Thomas M. Moerland

Pierre Gentine

Gustau Camps-Valls

2019-12-31

International Conference on Machine Learning (published)

Meta Attention Networks: Meta Learning Attention To Modulate Information Between Sparsely Interacting Recurrent Modules

Kanika Madan

Nan Rosemary Ke

Anirudh Goyal

Decomposing knowledge into interchangeable pieces promises a generalization advantage when, at some level of representation, the learner is … (see more)likely to be faced with situations requiring novel combinations of existing pieces of knowledge or computation. We hypothesize that such a decomposition of knowledge is particularly relevant for higher levels of representation as we see this at work in human cognition and natural language in the form of systematicity or systematic generalization. To study these ideas, we propose a particular training framework in which we assume that the pieces of knowledge an agent needs, as well as its reward function are stationary and can be re-used across tasks and changes in distribution. As the learner is confronted with variations in experiences, the attention selects which modules should be adapted and the parameters of those selected modules are adapted fast, while the parameters of attention mechanisms are updated slowly as meta-parameters. We ﬁnd that both the meta-learning and the modular aspects of the proposed system greatly help achieve faster learning in experiments with reinforcement learning setup involving navigation in a partially observed grid world.

2019-12-31

(published)

A Meta-Transfer Objective for Learning to Disentangle Causal Mechanisms

Tristan Deleu

Nasim Rahaman

Nan Rosemary Ke

Sébastien Lachapelle

Olexa Bilaniuk

Anirudh Goyal

Christopher Pal

We propose to meta-learn causal structures based on how fast a learner adapts to new distributions arising from sparse distributional change… (see more)s, e.g. due to interventions, actions of agents and other sources of non-stationarities. We show that under this assumption, the correct causal structural choices lead to faster adaptation to modified distributions because the changes are concentrated in one or just a few mechanisms when the learned knowledge is modularized appropriately. This leads to sparse expected gradients and a lower effective number of degrees of freedom needing to be relearned while adapting to the change. It motivates using the speed of adaptation to a modified distribution as a meta-learning objective. We demonstrate how this can be used to determine the cause-effect relationship between two observed variables. The distributional changes do not need to correspond to standard interventions (clamping a variable), and the learner has no direct knowledge of these interventions. We show that causal structures can be parameterized via continuous variables and learned end-to-end. We then explore how these ideas could be used to also learn an encoder that would map low-level observed variables to unobserved causal variables leading to faster adaptation out-of-distribution, learning a representation space where one can satisfy the assumptions of independent mechanisms and of small and sparse changes in these mechanisms due to actions and non-stationarities.

2019-12-31

ICLR (published)

N-BEATS: Neural Basis Expansion Analysis for Interpretable Time Series Forecasting

Boris N. Oreshkin

Dmitri Carpov

Nicolas Chapados

We focus on solving the univariate times series point forecasting problem using deep learning. We propose a deep neural architecture based o… (see more)n backward and forward residual links and a very deep stack of fully-connected layers. The architecture has a number of desirable properties, being interpretable, applicable without modification to a wide array of target domains, and fast to train. We test the proposed architecture on several well-known datasets, including M3, M4 and TOURISM competition datasets containing time series from diverse domains. We demonstrate state-of-the-art performance for two configurations of N-BEATS for all the datasets, improving forecast accuracy by 11% over a statistical benchmark and by 3% over last year's winner of the M4 competition, a domain-adjusted hand-crafted hybrid between neural network and statistical time series models. The first configuration of our model does not employ any time-series-specific components and its performance on heterogeneous datasets strongly suggests that, contrarily to received wisdom, deep learning primitives such as residual blocks are by themselves sufficient to solve a wide range of forecasting problems. Finally, we demonstrate how the proposed architecture can be augmented to provide outputs that are interpretable without considerable loss in accuracy.

2019-12-31

ICLR (published)