Yoshua Bengio

Biography

*For media requests, please write to medias@mila.quebec.

For more information please contact Julie Mongeau, executive assistant at julie.mongeau@mila.quebec.

Yoshua Bengio is recognized worldwide as a leading expert in AI. He is most known for his pioneering work in deep learning, which earned him the 2018 A.M. Turing Award, “the Nobel Prize of computing,” with Geoffrey Hinton and Yann LeCun.

Bengio is a full professor at Université de Montréal, and the founder and scientific director of Mila – Quebec Artificial Intelligence Institute. He is also a senior fellow at CIFAR and co-directs its Learning in Machines & Brains program, serves as scientific director of IVADO, and holds a Canada CIFAR AI Chair.

In 2019, Bengio was awarded the prestigious Killam Prize and in 2022, he was the most cited computer scientist in the world by h-index. He is a Fellow of the Royal Society of London, Fellow of the Royal Society of Canada, Knight of the Legion of Honor of France and Officer of the Order of Canada. In 2023, he was appointed to the UN’s Scientific Advisory Board for Independent Advice on Breakthroughs in Science and Technology.

Concerned about the social impact of AI, Bengio helped draft the Montréal Declaration for the Responsible Development of Artificial Intelligence and continues to raise awareness about the importance of mitigating the potentially catastrophic risks associated with future AI systems.

Current Students

Singh Aasheesh

Professional Master's

Research Intern - McGill University

Mohammed Abukalam

Research Intern - Université de Montréal

Rim Assouel

PhD - Université de Montréal

Collaborating Alumni

Research Intern - Université du Québec à Rimouski

Aayush Bajaj

Professional Master's - Université de Montréal

Stefan Bauer

Independent visiting researcher

Co-supervisor :

Guillaume Lajoie

Loubna Benabbou

Independent visiting researcher - UQAR

Paul Bertin

PhD - Université de Montréal

Ghait Boukachab

Research Intern - UQAR

Oussama Boussif

PhD - Université de Montréal

Independent visiting researcher - MIT

Xiaoyin Chen

PhD - Université de Montréal

Chen Chen

Postdoctorate - Université de Montréal

Co-supervisor :

Blake Richards

Pierre-Paul De Breuck

Collaborating Alumni - Université de Montréal

PhD - Université de Montréal

PhD - Université de Montréal

Collaborating researcher - Université Paris-Saclay

Principal supervisor :

Eric Elmoznino

PhD - Université de Montréal

Co-supervisor :

PhD - Université de Montréal

Katie Everett

PhD - Massachusetts Institute of Technology

Léna Nehale Ezzine

PhD - Université de Montréal

Jean-pierre Falet

PhD - Université de Montréal

Co-supervisor :

Leo Feng

PhD - Université de Montréal

PhD - Barcelona University

Piotr Gainski

Research Intern - Université de Montréal

Ahmad Ghawanmeh

Professional Master's - Université de Montréal

Ivan Grega Grega

Collaborating researcher - Université de Montréal

Pietro Greiner

Collaborating researcher

Mohsin Hasan

PhD - Université de Montréal

mohsin.hasan@mila.quebec

Alex Hernandez-Garcia

Postdoctorate - Université de Montréal

Co-supervisor :

Leon Hetzel

Independent visiting researcher - Technical University Munich (TUM)

Edward Hu

PhD - Université de Montréal

Moksh Jain

PhD - Université de Montréal

moksh.jain@mila.quebec

Research Intern - Université de Montréal

Master's Research - Université de Montréal

Co-supervisor :

Research Intern - Université de Montréal

Minsu Kim

Collaborating researcher - Université de Montréal

PhD - Université de Montréal

Michał Koziarski

Postdoctorate - Université de Montréal

Salem Lahlou

PhD - Université de Montréal

Hae-Beom Lee

Collaborating Alumni

Seanie Lee

Research Intern - Université de Montréal

Collaborating Alumni

Zhen Liu

PhD - Université de Montréal

Principal supervisor :

Liam Paull

Stephen Lu

Research Intern - McGill University

stephen.lu@mila.quebec

Research Intern - Imperial College London

PhD - Université de Montréal

Mohammed Mahfoud

Research Intern - Université de Montréal

Nikolay Malkin

Collaborating Alumni - Université de Montréal

DESS - Université de Montréal

Cristian Dragos Manta

PhD - Université de Montréal

Co-supervisor :

Dhanya Sridhar

Stefano Massaroli

Postdoctorate - Université de Montréal

Collaborating Alumni

Collaborating researcher - Université de Montréal

Sarthak Mittal

PhD - Université de Montréal

Principal supervisor :

PhD - Université de Montréal

Principal supervisor :

Independent visiting researcher - Université de Montréal

Ling Pan

Independent visiting researcher - Hong Kong University of Science and Technology (HKUST)

Ali Parviz

Collaborating researcher - Ying Wu Coll of Computing

Lena Podina

PhD - University of Waterloo

Principal supervisor :

Sarath Chandar Anbil Parthipan

Nassim Rahaman

PhD - Max-Planck-Institute for Intelligent Systems

Jarrid Rector-Brooks

PhD - Université de Montréal

Co-supervisor :

Danyal REHMAN

Postdoctorate - Université de Montréal

James Requeima

Independent visiting researcher - Université de Montréal

Jessie Richter-Powell

Independent visiting researcher - Université de Montréal

Camille Rochefort-Boulanger

PhD - Université de Montréal

Principal supervisor :

Julie Hussin

Theo Saulus

Collaborating researcher

Principal supervisor :

Victor Schmidt

PhD - Université de Montréal

Postdoctorate - Université de Montréal

Master's Research - Université de Montréal

Marcin Sendera

Research Intern - Université de Montréal

Dounia Shaaban Kabakibo

Research Intern - Université de Montréal

Vedant Shah

Master's Research - Université de Montréal

Collaborating Alumni

Marco Stock

Independent visiting researcher - Technical University of Munich

marco.stock@tum.de

Anja Surina

PhD - École Polytechnique Montréal Fédérale de Lausanne

Vincent Taboga

Postdoctorate - Polytechnique Montréal

Co-supervisor :

Pierre-Luc Bacon

Mélisande Astrid Crystal Teng

PhD - Université de Montréal

Co-supervisor :

Collaborating researcher

Principal supervisor :

alexander.tong@mila.quebec

Alex Tong

Postdoctorate - Université de Montréal

Collaborating researcher - Valence

Principal supervisor :

Dominique Beaini

Donna Vakalis

Postdoctorate - Université de Montréal

Co-supervisor :

Todosijevic Viktor Todosijevic

Collaborating researcher - RWTH Aachen University (Rheinisch-Westfälische Technische Hochschule Aachen)

Principal supervisor :

Sasha Volokhova

PhD - Université de Montréal

Zichao Yan

Collaborating Alumni - Université de Montréal

Elmimouni Zakaria

Research Intern - Université de Montréal

Dinghuai Zhang

PhD - Université de Montréal

Principal supervisor :

Aaron Courville

Ruixiang Zhang

PhD - Université de Montréal

Principal supervisor :

Liam Paull

William Zhang

PhD - Université de Montréal

PhD - McGill University

Principal supervisor :

Mathieu Blanchette

Skipper: Combining Spatial and Temporal Abstraction for Better Generalization

Harry Zhao

PhD - McGill University

Principal supervisor :

Blog Posts

Generic thumbnail for Mila Blog articles.

February 22, 2024

Mingde Harry Zhao

Safa Alver

Harm van Seijen

Romain Laroche

Doina Precup

Yoshua Bengio

Scaling in the Service of Reasoning & Model-Based ML

April 4, 2023

Yoshua Bengio

Edward J. Hu

A collaboration between Mila and Relation Therapeutics to discover novel synergistic combinations of drugs in vitro

March 23, 2022

Paul Bertin

Jake P. Taylor-King

Yoshua Bengio

March 15, 2022

Generative Flow Networks

Yoshua Bengio

Publications

Can a Bayesian Oracle Prevent Harm from an Agent?

Michael K. Cohen

Nikolay Malkin

Matt MacDermott

Damiano Fornasiere

Pietro Greiner

Younesse Kaddar

Is there a way to design powerful AI systems based on machine learning methods that would satisfy probabilistic safety guarantees? With the … (see more)long-term goal of obtaining a probabilistic guarantee that would apply in every context, we consider estimating a context-dependent bound on the probability of violating a given safety specification. Such a risk evaluation would need to be performed at run-time to provide a guardrail against dangerous actions of an AI. Noting that different plausible hypotheses about the world could produce very different outcomes, and because we do not know which one is right, we derive bounds on the safety violation probability predicted under the true but unknown hypothesis. Such bounds could be used to reject potentially dangerous actions. Our main results involve searching for cautious but plausible hypotheses, obtained by a maximization that involves Bayesian posteriors over hypotheses. We consider two forms of this result, in the iid case and in the non-iid case, and conclude with open problems towards turning such theoretical results into practical AI guardrails.

2024-08-09

ArXiv (preprint)

Open Problems in Technical AI Governance

Anka Reuel

Benjamin Bucknall

Stephen Casper

Tim Fist

Lisa Soder

Onni Aarne

Lewis Hammond

Lujain Ibrahim

Alan Chan

Peter Wills

Markus Anderljung

Ben Garfinkel

Lennart Heim

Andrew Trask

Gabriel Mukobi

Rylan Schaeffer

Mauricio Baker

Sara Hooker

Irene Solaiman

Alexandra Luccioni … (see 11 more)

Nitarshan Rajkumar

Nicolas Moes

Jeffrey Ladish

Neel Guha

Jessica Newman

Tobin South

Alex Pentland

Sanmi Koyejo

Mykel J. Kochenderfer

Robert F. Trager

AI progress is creating a growing range of risks and opportunities, but it is often unclear how they should be navigated. In many cases, the… (see more) barriers and uncertainties faced are at least partly technical. Technical AI governance, referring to technical analysis and tools for supporting the effective governance of AI, seeks to address such challenges. It can help to (a) identify areas where intervention is needed, (b) identify and assess the efficacy of potential governance actions, and (c) enhance governance options by designing mechanisms for enforcement, incentivization, or compliance. In this paper, we explain what technical AI governance is, why it is important, and present a taxonomy and incomplete catalog of its open problems. This paper is intended as a resource for technical researchers or research funders looking to contribute to AI governance.

2024-07-20

ArXiv (preprint)

Improving Gradient-Guided Nested Sampling for Posterior Inference

Pablo Lemos

Nikolay Malkin

Will Handley

Laurence Perreault-Levasseur

Yashar Hezaveh

We present a performant, general-purpose gradient-guided nested sampling (GGNS) algorithm, combining the state of the art in differentiable … (see more)programming, Hamiltonian slice sampling, clustering, mode separation, dynamic nested sampling, and parallelization. This unique combination allows GGNS to scale well with dimensionality and perform competitively on a variety of synthetic and real-world problems. We also show the potential of combining nested sampling with generative flow networks to obtain large amounts of high-quality samples from the posterior distribution. This combination leads to faster mode discovery and more accurate estimates of the partition function.

2024-07-08

Proceedings of the 41st International Conference on Machine Learning (published)

Memory Efficient Neural Processes via Constant Memory Attention Block

Leo Feng

Frederick Tung

Hossein Hajimirsadeghi

Mohamed Osama Ahmed

2024-07-08

Proceedings of the 41st International Conference on Machine Learning (published)

proceedings.mlr.press

On Generalization for Generative Flow Networks

Anas Krichel

Nikolay Malkin

Salem Lahlou

Generative Flow Networks (GFlowNets) have emerged as an innovative learning paradigm designed to address the challenge of sampling from an u… (see more)nnormalized probability distribution, called the reward function. This framework learns a policy on a constructed graph, which enables sampling from an approximation of the target probability distribution through successive steps of sampling from the learned policy. To achieve this, GFlowNets can be trained with various objectives, each of which can lead to the model s ultimate goal. The aspirational strength of GFlowNets lies in their potential to discern intricate patterns within the reward function and their capacity to generalize effectively to novel, unseen parts of the reward function. This paper attempts to formalize generalization in the context of GFlowNets, to link generalization with stability, and also to design experiments that assess the capacity of these models to uncover unseen parts of the reward function. The experiments will focus on length generalization meaning generalization to states that can be constructed only by longer trajectories than those seen in training.

2024-07-03

ArXiv (preprint)

Meta Flow Matching: Integrating Vector Fields on the Wasserstein Manifold

Lazar Atanackovic

Xi Zhang

Brandon Amos

Mathieu Blanchette

Leo J Lee

Alexander Tong

Kirill Neklyudov

Numerous biological and physical processes can be modeled as systems of interacting samples evolving continuously over time, e.g. the dynami… (see more)cs of communicating cells or physical particles. Flow-based models allow for learning these dynamics at the population level --- they model the evolution of the entire distribution of samples. However, current flow-based models are limited to a single initial population and a set of predefined conditions which describe different dynamics. We propose

2024-06-17

ICML.cc/2024/Workshop/GRaM (published)

RGFN: Synthesizable Molecular Generation Using GFlowNets

Michał Koziarski

Andrei Rekesh

Dmytro Shevchuk

Almer M. van der Sloot

Piotr Gaiński

Cheng-Hao Liu

Mike Tyers

Robert A. Batey

2024-06-17

ICML.cc/2024/Workshop/ML4LMS (poster)

Cell Morphology-Guided Small Molecule Generation with GFlowNets

Stephen Zhewen Lu

Ziqing Lu

Ehsan Hajiramezanali

Tommaso Biancalani

Gabriele Scalia

Michał Koziarski

2024-06-16

ICML.cc/2024/Workshop/AI4Science (poster)

AI-Assisted Generation of Difficult Math Questions

Vedant Shah

Dingli Yu

Kaifeng Lyu

Simon Park

Nan Rosemary Ke

Michael Curtis Mozer

James Lloyd McClelland

Sanjeev Arora

Anirudh Goyal

Current LLM training positions mathematical reasoning as a core capability. With publicly available sources fully tapped, there is unmet dem… (see more)and for diverse and challenging math questions. Relying solely on human experts is both time-consuming and costly, while LLM-generated questions often lack the requisite diversity and difficulty. We present a design framework that combines the strengths of LLMs with a human-in-the-loop approach to generate a diverse array of challenging math questions. We leverage LLM metacognition skills [Didolkar et al., 2024] of a strong LLM to extract core"skills"from existing math datasets. These skills serve as the basis for generating novel and difficult questions by prompting the LLM with random pairs of core skills. The use of two different skills within each question makes finding such questions an"out of distribution"task for both LLMs and humans. Our pipeline employs LLMs to iteratively generate and refine questions and solutions through multiturn prompting. Human annotators then verify and further refine the questions, with their efficiency enhanced via further LLM interactions. Applying this pipeline on skills extracted from the MATH dataset [Hendrycks et al., 2021] resulted in MATH

2024-06-13

ICML.cc/2024/Workshop/AI4MATH (poster)

Metacognitive Capabilities of LLMs: An Exploration in Mathematical Problem Solving

Aniket Rajiv Didolkar

Anirudh Goyal

Nan Rosemary Ke

Siyuan Guo

Michal Valko

Timothy P Lillicrap

Danilo Jimenez Rezende

Michael Curtis Mozer

Sanjeev Arora

2024-06-13

ICML.cc/2024/Workshop/AI4MATH (poster)

MAP: Low-compute Model Merging with Amortized Pareto Fronts via Quadratic Approximation

Lu Li

Tianyu Zhang

Zhiqi Bu

Suyuchen Wang

Huan He

Jie Fu

Yonghui Wu

Jiang Bian

Yong Chen

Model merging has emerged as an effective approach to combine multiple single-task models, fine-tuned from the same pre-trained model, into … (see more)a multitask model. This process typically involves computing a weighted average of the model parameters without any additional training. Existing model-merging methods focus on enhancing average task accuracy. However, interference and conflicts between the objectives of different tasks can lead to trade-offs during model merging. In real-world applications, a set of solutions with various trade-offs can be more informative, helping practitioners make decisions based on diverse preferences. In this paper, we introduce a novel low-compute algorithm, Model Merging with Amortized Pareto Front (MAP). MAP identifies a Pareto set of scaling coefficients for merging multiple models to reflect the trade-offs. The core component of MAP is approximating the evaluation metrics of the various tasks using a quadratic approximation surrogate model derived from a pre-selected set of scaling coefficients, enabling amortized inference. Experimental results on vision and natural language processing tasks show that MAP can accurately identify the Pareto front. To further reduce the required computation of MAP, we propose (1) a Bayesian adaptive sampling algorithm and (2) a nested merging scheme with multiple stages.

2024-06-11

ArXiv (preprint)

VCR: Visual Caption Restoration

Tianyu Zhang

Suyuchen Wang

Lu Li

Ge Zhang

Perouz Taslakian

Sai Rajeswar

Jie Fu

Bang Liu

We introduce Visual Caption Restoration (VCR), a novel vision-language task that challenges models to accurately restore partially obscured … (see more)texts using pixel-level hints within images. This task stems from the observation that text embedded in images is intrinsically different from common visual elements and natural language due to the need to align the modalities of vision, text, and text embedded in images. While numerous works have integrated text embedded in images into visual question-answering tasks, approaches to these tasks generally rely on optical character recognition or masked language modeling, thus reducing the task to mainly text-based processing. However, text-based processing becomes ineffective in VCR as accurate text restoration depends on the combined information from provided images, context, and subtle cues from the tiny exposed areas of masked texts. We develop a pipeline to generate synthetic images for the VCR task using image-caption pairs, with adjustable caption visibility to control the task difficulty. With this pipeline, we construct a dataset for VCR called VCR-Wiki using images with captions from Wikipedia, comprising 2.11M English and 346K Chinese entities in both easy and hard split variants. Our results reveal that current vision language models significantly lag behind human performance in the VCR task, and merely fine-tuning the models on our dataset does not lead to notable improvements. We release VCR-Wiki and the data construction code to facilitate future research.

2024-06-10

ArXiv (preprint)