Yoshua Bengio

Biography

*For media requests, please write to medias@mila.quebec.

For more information please contact Julie Mongeau, executive assistant at julie.mongeau@mila.quebec.

Yoshua Bengio is recognized worldwide as a leading expert in AI. He is most known for his pioneering work in deep learning, which earned him the 2018 A.M. Turing Award, “the Nobel Prize of computing,” with Geoffrey Hinton and Yann LeCun.

Bengio is a full professor at Université de Montréal, and the founder and scientific director of Mila – Quebec Artificial Intelligence Institute. He is also a senior fellow at CIFAR and co-directs its Learning in Machines & Brains program, serves as scientific director of IVADO, and holds a Canada CIFAR AI Chair.

In 2019, Bengio was awarded the prestigious Killam Prize and in 2022, he was the most cited computer scientist in the world by h-index. He is a Fellow of the Royal Society of London, Fellow of the Royal Society of Canada, Knight of the Legion of Honor of France and Officer of the Order of Canada. In 2023, he was appointed to the UN’s Scientific Advisory Board for Independent Advice on Breakthroughs in Science and Technology.

Concerned about the social impact of AI, Bengio helped draft the Montréal Declaration for the Responsible Development of Artificial Intelligence and continues to raise awareness about the importance of mitigating the potentially catastrophic risks associated with future AI systems.

Current Students

Jamal Abou Haibeh

Research Intern - McGill University

Mohammed Abukalam

Research Intern - Université de Montréal

Rim Assouel

PhD - Université de Montréal

Collaborating Alumni

Research Intern - Université du Québec à Rimouski

Stefan Bauer

Independent visiting researcher

Co-supervisor :

Guillaume Lajoie

Paul Bertin

PhD - Université de Montréal

Ghait Boukachab

Research Intern - UQAR

Oussama Boussif

PhD - Université de Montréal

Independent visiting researcher - MIT

Shahana Chatterjee

Collaborating researcher - N/A

Principal supervisor :

Chen Chen

Postdoctorate - Université de Montréal

Co-supervisor :

Blake Richards

Xiaoyin Chen

PhD - Université de Montréal

Pierre-Paul De Breuck

Collaborating Alumni - Université de Montréal

PhD - Université de Montréal

PhD - Université de Montréal

Collaborating researcher - Université Paris-Saclay

Principal supervisor :

Eric Elmoznino

PhD - Université de Montréal

Co-supervisor :

PhD - Université de Montréal

Katie Everett

PhD - Massachusetts Institute of Technology

Léna Nehale Ezzine

PhD - Université de Montréal

Jean-Pierre Falet

PhD - Université de Montréal

Co-supervisor :

Leo Feng

PhD - Université de Montréal

Research Intern - Barcelona University

Piotr Gainski

Research Intern - Université de Montréal

Ivan Grega

Collaborating researcher - Université de Montréal

Pietro Greiner

Research Intern

Mohsin Hasan

PhD - Université de Montréal

mohsin.hasan@mila.quebec

Alex Hernandez-Garcia

Postdoctorate - Université de Montréal

Co-supervisor :

Leon Hetzel

Independent visiting researcher - Technical University Munich (TUM)

Edward Hu

PhD - Université de Montréal

Moksh Jain

PhD - Université de Montréal

moksh.jain@mila.quebec

Research Intern - Université de Montréal

Master's Research - Université de Montréal

Co-supervisor :

Research Intern - Université de Montréal

Minsu Kim

Collaborating researcher - Université de Montréal

PhD - Université de Montréal

Michał Koziarski

Postdoctorate - Université de Montréal

Salem Lahlou

PhD - Université de Montréal

Hae-Beom Lee

Collaborating Alumni

Seanie Lee

Collaborating Alumni - Université de Montréal

Collaborating Alumni

Zhen Liu

PhD - Université de Montréal

Principal supervisor :

Liam Paull

Matt MacDermott

Research Intern - Imperial College London

PhD - Université de Montréal

Mohammed Mahfoud

Research Intern - Université de Montréal

Nikolay Malkin

Collaborating Alumni - Université de Montréal

Cristian Dragos Manta

PhD - Université de Montréal

Co-supervisor :

Dhanya Sridhar

Stefano Massaroli

Postdoctorate - Université de Montréal

Collaborating Alumni

Collaborating researcher - Université de Montréal

Sarthak Mittal

PhD - Université de Montréal

Principal supervisor :

PhD - Université de Montréal

Principal supervisor :

Postdoctorate - Université de Montréal

Principal supervisor :

Independent visiting researcher - Université de Montréal

Ling Pan

Independent visiting researcher - Hong Kong University of Science and Technology (HKUST)

Ali Parviz

Collaborating researcher - Ying Wu Coll of Computing

Lena Podina

PhD - University of Waterloo

Principal supervisor :

Nassim Rahaman

PhD - Max-Planck-Institute for Intelligent Systems

Jarrid Rector-Brooks

PhD - Université de Montréal

Co-supervisor :

Sarath Chandar

Danyal REHMAN

Postdoctorate - Université de Montréal

James Requeima

Independent visiting researcher - Université de Montréal

Oli RICHARDSON

Postdoctorate - Université de Montréal

Jessie Richter-Powell

Independent visiting researcher - Université de Montréal

Camille Rochefort-Boulanger

PhD - Université de Montréal

Principal supervisor :

Julie Hussin

agassoussisalwane2@gmail.com

Salwane Salwane

Research Intern - Université de Montréal

Theo Saulus

Collaborating researcher

Principal supervisor :

Victor Schmidt

PhD - Université de Montréal

Postdoctorate - Université de Montréal

Master's Research - Université de Montréal

Marcin Sendera

Research Intern - Université de Montréal

Dounia Shaaban Kabakibo

Research Intern - Université de Montréal

Vedant Shah

Master's Research - Université de Montréal

Collaborating Alumni

Marco Stock

Independent visiting researcher - Technical University of Munich

marco.stock@tum.de

Anja Surina

PhD - École Polytechnique Montréal Fédérale de Lausanne

Vincent Taboga

Postdoctorate - Polytechnique Montréal

Co-supervisor :

Pierre-Luc Bacon

Mélisande Astrid Crystal Teng

PhD - Université de Montréal

Co-supervisor :

Collaborating researcher

Principal supervisor :

alexander.tong@mila.quebec

Alex Tong

Postdoctorate - Université de Montréal

Collaborating researcher - Valence

Principal supervisor :

Dominique Beaini

Donna Vakalis

Postdoctorate - Université de Montréal

Co-supervisor :

Viktor Viktor Todosijevic

Collaborating researcher - RWTH Aachen University (Rheinisch-Westfälische Technische Hochschule Aachen)

Principal supervisor :

Sasha Volokhova

PhD - Université de Montréal

Zichao Yan

Collaborating Alumni - Université de Montréal

Kyle YUN

Collaborating researcher - KAIST

Elmimouni Zakaria

Research Intern - Université de Montréal

Nicole Zhang

PhD - McGill University

Principal supervisor :

Mathieu Blanchette

Dinghuai Zhang

PhD - Université de Montréal

Principal supervisor :

Aaron Courville

Ruixiang Zhang

PhD - Université de Montréal

Principal supervisor :

Liam Paull

Tianyu Zhang

PhD - Université de Montréal

Skipper: Combining Spatial and Temporal Abstraction for Better Generalization

Harry Zhao

PhD - McGill University

Principal supervisor :

Blog Posts

Generic thumbnail for Mila Blog articles.

February 22, 2024

Mingde Harry Zhao

Safa Alver

Harm van Seijen

Romain Laroche

Doina Precup

Yoshua Bengio

Scaling in the Service of Reasoning & Model-Based ML

April 4, 2023

Yoshua Bengio

Edward J. Hu

A collaboration between Mila and Relation Therapeutics to discover novel synergistic combinations of drugs in vitro

March 23, 2022

Paul Bertin

Jake P. Taylor-King

Yoshua Bengio

March 15, 2022

Generative Flow Networks

Yoshua Bengio

Publications

SatBird: Bird Species Distribution Modeling with Remote Sensing and Citizen Science Data

Mélisande Teng

Amna Elmustafa

Benjamin Akera

Hager Radi

Hugo Larochelle

Biodiversity is declining at an unprecedented rate, impacting ecosystem services necessary to ensure food, water, and human health and well-… (see more)being. Understanding the distribution of species and their habitats is crucial for conservation policy planning. However, traditional methods in ecology for species distribution models (SDMs) generally focus either on narrow sets of species or narrow geographical areas and there remain significant knowledge gaps about the distribution of species. A major reason for this is the limited availability of data traditionally used, due to the prohibitive amount of effort and expertise required for traditional field monitoring. The wide availability of remote sensing data and the growing adoption of citizen science tools to collect species observations data at low cost offer an opportunity for improving biodiversity monitoring and enabling the modelling of complex ecosystems. We introduce a novel task for mapping bird species to their habitats by predicting species encounter rates from satellite images, and present SatBird, a satellite dataset of locations in the USA with labels derived from presence-absence observation data from the citizen science database eBird, considering summer (breeding) and winter seasons. We also provide a dataset in Kenya representing low-data regimes. We additionally provide environmental data and species range maps for each location. We benchmark a set of baselines on our dataset, including SOTA models for remote sensing tasks. SatBird opens up possibilities for scalably modelling properties of ecosystems worldwide.

2023-11-02

ArXiv (preprint)

arxiv.org

Generative AI models should include detection mechanisms as a condition for public release

Alistair Knott

Dino Pedreschi

Raja Chatila

Tapabrata Chakraborti

Susan Leavy

Ricardo Baeza-Yates

D. Eyers

Andrew Trotman

Paul D. Teal

Przemyslaw Biecek

Stuart Russell

2023-10-28

Ethics and Information Technology (published)

OC-NMN: Object-centric Compositional Neural Module Network for Generative Visual Analogical Reasoning

Rim Assouel

Pau Rodriguez

Perouz Taslakian

David Vazquez

2023-10-28

ArXiv (preprint)

arxiv.org

Attention Schema in Neural Agents

Dianbo Liu

Samuele Bolotta

Mike He Zhu

Zahra Sheikhbahaee

Guillaume Dumas

Attention has become a common ingredient in deep learning architectures. It adds a dynamical selection of information on top of the static s… (see more)election of information supported by weights. In the same way, we can imagine a higher-order informational filter built on top of attention: an Attention Schema (AS), namely, a descriptive and predictive model of attention. In cognitive neuroscience, Attention Schema Theory (AST) supports this idea of distinguishing attention from AS. A strong prediction of this theory is that an agent can use its own AS to also infer the states of other agents' attention and consequently enhance coordination with other agents. As such, multi-agent reinforcement learning would be an ideal setting to experimentally test the validity of AST. We explore different ways in which attention and AS interact with each other. Our preliminary results indicate that agents that implement the AS as a recurrent internal control achieve the best performance. In general, these exploratory experiments suggest that equipping artificial agents with a model of attention can enhance their social intelligence.

2023-10-27

NeurIPS.cc/2023/Workshop/InfoCog (poster)

Baking Symmetry into GFlowNets

George Ma

Emmanuel Bengio

Dinghuai Zhang

GFlowNets have exhibited promising performance in generating diverse candidates with high rewards. These networks generate objects increment… (see more)ally and aim to learn a policy that assigns probability of sampling objects in proportion to rewards. However, the current training pipelines of GFlowNets do not consider the presence of isomorphic actions, which are actions resulting in symmetric or isomorphic states. This lack of symmetry increases the amount of samples required for training GFlowNets and can result in inefficient and potentially incorrect flow functions. As a consequence, the reward and diversity of the generated objects decrease. In this study, our objective is to integrate symmetries into GFlowNets by identifying equivalent actions during the generation process. Experimental results using synthetic data demonstrate the promising performance of our proposed approaches.

2023-10-27

NeurIPS.cc/2023/Workshop/AI4Science (oral)

Baking Symmetry into GFlowNets

George Ma

Emmanuel Bengio

Dinghuai Zhang

2023-10-27

NeurIPS.cc/2023/Workshop/AI4Science (oral)

Causal Discovery in Gene Regulatory Networks with GFlowNet: Towards Scalability in Large Systems

Trang Nguyen

Alexander Tong

Kanika Madan

Dianbo Liu

Understanding causal relationships within Gene Regulatory Networks (GRNs) is essential for unraveling the gene interactions in cellular proc… (see more)esses. However, causal discovery in GRNs is a challenging problem for multiple reasons including the existence of cyclic feedback loops and uncertainty that yields diverse possible causal structures. Previous works in this area either ignore cyclic dynamics (assume acyclic structure) or struggle with scalability. We introduce Swift-DynGFN as a novel framework that enhances causal structure learning in GRNs while addressing scalability concerns. Specifically, Swift-DynGFN exploits gene-wise independence to boost parallelization and to lower computational cost. Experiments on real single-cell RNA velocity and synthetic GRN datasets showcase the advancement in learning causal structure in GRNs and scalability in larger systems.

2023-10-27

NeurIPS.cc/2023/Workshop/GenBio (poster)

Crystal-GFN: sampling materials with desirable properties and constraints

Mistal

Alex Hernandez-Garcia

Alexandra Volokhova

Alexandre AGM Duval

Divya Sharma

pierre luc carrier

Michał Koziarski

Victor Schmidt

2023-10-27

NeurIPS.cc/2023/Workshop/AI4Mat (spotlight)

Discrete, compositional, and symbolic representations through attractor dynamics

Andrew Nam

Eric Elmoznino

Nikolay Malkin

Chen Sun

Guillaume Lajoie

Compositionality is an important feature of discrete symbolic systems, such as language and programs, as it enables them to have infinite ca… (see more)pacity despite a finite symbol set. It serves as a useful abstraction for reasoning in both cognitive science and in AI, yet the interface between continuous and symbolic processing is often imposed by fiat at the algorithmic level, such as by means of quantization or a softmax sampling step. In this work, we explore how discretization could be implemented in a more neurally plausible manner through the modeling of attractor dynamics that partition the continuous representation space into basins that correspond to sequences of symbols. Building on established work in attractor networks and introducing novel training methods, we show that imposing structure in the symbolic space can produce compositionality in the attractor-supported representation space of rich sensory inputs. Lastly, we argue that our model exhibits the process of an information bottleneck that is thought to play a role in conscious experience, decomposing the rich information of a sensory input into stable components encoding symbolic information.

2023-10-27

NeurIPS.cc/2023/Workshop/InfoCog (oral)

Learning to Scale Logits for Temperature-Conditional GFlowNets

Minsu Kim

Joohwan Ko

Dinghuai Zhang

Ling Pan

Taeyoung Yun

Woo Chang Kim

Jinkyoo Park

GFlowNets are probabilistic models that learn a stochastic policy that sequentially generates compositional structures, such as molecular gr… (see more)aphs. They are trained with the objective of sampling such objects with probability proportional to the object's reward. Among GFlowNets, the temperature-conditional GFlowNets represent a family of policies indexed by temperature, and each is associated with the correspondingly tempered reward function. The major benefit of temperature-conditional GFlowNets is the controllability of GFlowNets' exploration and exploitation through adjusting temperature. We propose a \textit{Learning to Scale Logits for temperature-conditional GFlowNets} (LSL-GFN), a novel architectural design that greatly accelerates the training of temperature-conditional GFlowNets. It is based on the idea that previously proposed temperature-conditioning approaches introduced numerical challenges in the training of the deep network because different temperatures may give rise to very different gradient profiles and ideal scales of the policy's logits. We find that the challenge is greatly reduced if a learned function of the temperature is used to scale the policy's logits directly. We empirically show that our strategy dramatically improves the performances of GFlowNets, outperforming other baselines, including reinforcement learning and sampling methods, in terms of discovering diverse modes in multiple biochemical tasks.

2023-10-27

NeurIPS.cc/2023/Workshop/AI4Science (poster)

Multi-Fidelity Active Learning with GFlowNets

Alex Hernandez-Garcia

Nikita Saxena

Moksh J. Jain

Cheng-Hao Liu

In the last decades, the capacity to generate large amounts of data in science and engineering applications has been growing steadily. Meanw… (see more)hile, the progress in machine learning has turned it into a suitable tool to process and utilise the available data. Nonetheless, many relevant scientific and engineering problems present challenges where current machine learning methods cannot yet efficiently leverage the available data and resources. For example, in scientific discovery, we are often faced with the problem of exploring very large, high-dimensional spaces, where querying a high fidelity, black-box objective function is very expensive. Progress in machine learning methods that can efficiently tackle such problems would help accelerate currently crucial areas such as drug and materials discovery. In this paper, we propose the use of GFlowNets for multi-fidelity active learning, where multiple approximations of the black-box function are available at lower fidelity and cost. GFlowNets are recently proposed methods for amortised probabilistic inference that have proven efficient for exploring large, high-dimensional spaces and can hence be practical in the multi-fidelity setting too. Here, we describe our algorithm for multi-fidelity active learning with GFlowNets and evaluate its performance in both well-studied synthetic tasks and practically relevant applications of molecular discovery. Our results show that multi-fidelity active learning with GFlowNets can efficiently leverage the availability of multiple oracles with different costs and fidelities to accelerate scientific discovery and engineering design.

2023-10-27

NeurIPS.cc/2023/Workshop/ReALML (published)

On the importance of catalyst-adsorbate 3D interactions for relaxed energy predictions

Alvaro Carbonero

Alexandre AGM Duval

Victor Schmidt

Santiago Miret

Alex Hernandez-Garcia

The use of machine learning for material property prediction and discovery has traditionally centered on graph neural networks that incorpor… (see more)ate the geometric configuration of all atoms. However, in practice not all this information may be readily available, e.g.~when evaluating the potentially unknown binding of adsorbates to catalyst. In this paper, we investigate whether it is possible to predict a system's relaxed energy in the OC20 dataset while ignoring the relative position of the adsorbate with respect to the electro-catalyst. We consider SchNet, DimeNet++ and FAENet as base architectures and measure the impact of four modifications on model performance: removing edges in the input graph, pooling independent representations, not sharing the backbone weights and using an attention mechanism to propagate non-geometric relative information. We find that while removing binding site information impairs accuracy as expected, modified models are able to predict relaxed energies with remarkably decent MAE. Our work suggests future research directions in accelerated materials discovery where information on reactant configurations can be reduced or altogether omitted.

2023-10-27

NeurIPS.cc/2023/Workshop/AI4Mat (poster)