Portrait of Yoshua Bengio

Yoshua Bengio

Core Academic Member

Canada CIFAR AI Chair

Full Professor, Université de Montréal, Department of Computer Science and Operations Research Department

Research Topics

Causality

Computational Neuroscience

Deep Learning

Generative Models

Graph Neural Networks

Machine Learning Theory

Medical Machine Learning

Molecular Modeling

Natural Language Processing

Probabilistic Models

Reasoning

Recurrent Neural Networks

Reinforcement Learning

Representation Learning

Biography

*For media requests, please write to medias@mila.quebec.

For more information please contact Cassidy MacNeil, Senior Assistant and Operation Lead at cassidy.macneil@mila.quebec.

Yoshua Bengio is recognized worldwide as a leading expert in AI. He is most known for his pioneering work in deep learning, which earned him the 2018 A.M. Turing Award, “the Nobel Prize of computing,” with Geoffrey Hinton and Yann LeCun.

Bengio is a full professor at Université de Montréal, and the founder and scientific advisor of Mila – Quebec Artificial Intelligence Institute. He is also a senior fellow at CIFAR and co-directs its Learning in Machines & Brains program, serves as special advisor and founding scientific director of IVADO, and holds a Canada CIFAR AI Chair.

In 2019, Bengio was awarded the prestigious Killam Prize and in 2022, he was the most cited computer scientist in the world by h-index. He is a Fellow of the Royal Society of London, Fellow of the Royal Society of Canada, Knight of the Legion of Honor of France and Officer of the Order of Canada. In 2023, he was appointed to the UN’s Scientific Advisory Board for Independent Advice on Breakthroughs in Science and Technology.

Concerned about the social impact of AI, Bengio helped draft the Montréal Declaration for the Responsible Development of Artificial Intelligence and continues to raise awareness about the importance of mitigating the potentially catastrophic risks associated with future AI systems.

Current Students

Jamal Abou Haibeh

Collaborating Alumni - McGill University

jamal.abouhaibeh@mila.quebec

Collaborating researcher - Cambridge University

Principal supervisor :

berkes.anais@mila.quebec

PhD - Université de Montréal

assouelr@mila.quebec

Shahana Chatterjee

Collaborating researcher - N/A

Principal supervisor :

shahana.chatterjee@mila.quebec

PhD - Université de Montréal

xiaoyin.chen@mila.quebec

Collaborating researcher - KAIST

sanghyeok.choi@mila.quebec

Aniket Didolkar

PhD - Université de Montréal

aniket.didolkar@mila.quebec

PhD - Université de Montréal

Co-supervisor :

Guillaume Lajoie

eric.elmoznino@mila.quebec

PhD - Université de Montréal

lena-nehale.ezzine@mila.quebec

Jean-Pierre Falet

PhD - Université de Montréal

jean-pierre.falet@mila.quebec

PhD

pietro.greiner@mila.quebec

PhD - Université de Montréal

mohsin.hasan@mila.quebec

PhD - Université de Montréal

moksh.jain@mila.quebec

Thomas Jiralerspong

PhD - Université de Montréal

Principal supervisor :

Guillaume Lajoie

thomas.jiralerspong@mila.quebec

Younesse Kaddar

Collaborating Alumni - Université de Montréal

younesse.kaddar@mila.quebec

Postdoctorate - Université de Montréal

Principal supervisor :

Alex Hernández-García

hyeonah.kim@mila.quebec

Collaborating researcher - Université de Montréal

minsu.kim@mila.quebec

Research Intern - KOREA ADVANCED INSTITUTE OF SCIENCE AND TECHNOLOGY

jeongjae.lee@mila.quebec

Collaborating researcher - s.o.

liusong299@gmail.com

Collaborating researcher

liucheng@mila.quebec

Cristian Dragos Manta

PhD - Université de Montréal

Co-supervisor :

cristian-dragos.manta@mila.quebec

PhD - Université de Montréal

Principal supervisor :

Guillaume Lajoie

mittalsa@mila.quebec

Independent visiting researcher - Université de Montréal

nguyenph@mila.quebec

Collaborating researcher - Ying Wu Coll of Computing

ali.parviz@mila.quebec

Collaborating researcher - University of Waterloo

Principal supervisor :

elena.podina@mila.quebec

Jarrid Rector-Brooks

PhD - Université de Montréal

jarrid.rector-brooks@mila.quebec

Postdoctorate - Université de Montréal

danyal.rehman@mila.quebec

Postdoctorate - Université de Montréal

oliver.richardson@mila.quebec

Camille Rochefort-Boulanger

PhD - Université de Montréal

Principal supervisor :

rochefoc@mila.quebec

Postdoctorate - Université de Montréal

Co-supervisor :

Alex Hernández-García

divya.sharma@mila.quebec

Mélisande Astrid Crystal Teng

Collaborating Alumni - Université de Montréal

Co-supervisor :

Hugo Larochelle

tengmeli@mila.quebec

Collaborating Alumni - Université de Montréal

alexander.tong@mila.quebec

Collaborating researcher - University of Cambridge

oszkar.urban@mila.quebec

Collaborating Alumni - Université de Montréal

Co-supervisor :

donna.vakalis@mila.quebec

Siddarth Venkatraman

PhD - Université de Montréal

Principal supervisor :

siddarth.venkatraman@mila.quebec

Esmeralda Whitammer

Collaborating researcher - Université de Montréal

esmeralda.whitammer@mila.quebec

Collaborating Alumni

omar.younis@mila.quebec

Collaborating researcher - Université de Montréal

taeyoung.yun@mila.quebec

PhD - Université de Montréal

tianyu.zhang@mila.quebec

PhD - McGill University

Principal supervisor :

Mathieu Blanchette

xi.zhang@mila.quebec

Blog Posts

Generic thumbnail for Mila Blog articles.

February 22, 2024

Skipper: Combining Spatial and Temporal Abstraction for Better Generalization

by

Mingde Harry Zhao

Safa Alver

Harm van Seijen

Romain Laroche

Read the article

Scaling in the service of reasoning & model-based ML

April 4, 2023

Scaling in the Service of Reasoning & Model-Based ML

by

Read the article

A collaboration between Mila and Relation Therapeutics to discover novel synergistic combinations of drugs in vitro

March 23, 2022

A collaboration between Mila and Relation Therapeutics to discover novel synergistic combinations of drugs in vitro

by

Jake P. Taylor-King

Read the article

Generative Flow Networks

March 15, 2022

Generative Flow Networks

by

Read the article

Publications

Language Models Can Reduce Asymmetry in Information Markets

Nasim Rahaman

Manuel Wuthrich

Erran L. Li

Christopher Pal

Bernhard Schölkopf

2024-03-20

ArXiv (preprint)

Improving and Generalizing Flow-Based Generative Models with Minibatch Optimal Transport

Nikolay Malkin

Guillaume Huguet

Yanlei Zhang

Jarrid Rector-Brooks

Continuous normalizing flows (CNFs) are an attractive generative modeling technique, but they have been held back by limitations in their si… (see more)mulation-based maximum likelihood training. We introduce the generalized \textit{conditional flow matching} (CFM) technique, a family of simulation-free training objectives for CNFs. CFM features a stable regression objective like that used to train the stochastic flow in diffusion models but enjoys the efficient inference of deterministic flow models. In contrast to both diffusion models and prior CNF training algorithms, CFM does not require the source distribution to be Gaussian or require evaluation of its density. A variant of our objective is optimal transport CFM (OT-CFM), which creates simpler flows that are more stable to train and lead to faster inference, as evaluated in our experiments. Furthermore, OT-CFM is the first method to compute dynamic OT in a simulation-free way. Training CNFs with CFM improves results on a variety of conditional and unconditional generation tasks, such as inferring single cell dynamics, unsupervised image translation, and Schrödinger bridge inference.

2024-03-10

TMLR (accepted)

Integrating Generative and Experimental Platforms or Biomolecular Design

Cheng-Hao Liu

Jarrid Rector-Brooks

Jason Yim

Soojung Yang

Sidney Lisanza

Francesca-Zhoufan Li

Pranam Chatterjee

Tommi Jaakkola

Regina Barzilay

David Baker

Frances H. Arnold

2024-03-07

ICLR.cc/2024/Workshop_Proposals (published)

Tackling Climate Change with Machine Learning: Fostering the Maturity of ML Applications for Climate Change

Shiva Madadkhani

Olivia Mendivil Ramos

Millie Chapman

Jesse Dunietz

Arthur Ouaknine

2024-03-07

ICLR.cc/2024/Workshop_Proposals (published)

Efficient Causal Graph Discovery Using Large Language Models

Thomas Jiralerspong

Yash More

2024-03-04

AGI @ International Conference on Learning Representations (poster)

Towards DNA-Encoded Library Generation with GFlowNets

Michał Koziarski

Mohammed Abukalam

Louis Vaillancourt

Doris Alexandra Schuetz

Almer Van Der Sloot

Mathieu Bourgey

Anne Marinier

2024-03-03

GEM @ International Conference on Learning Representations (poster)

Distributional GFlowNets with Quantile Flows

Ricky T. Q. Chen

Aaron Courville

Generative Flow Networks (GFlowNets) are a new family of probabilistic samplers where an agent learns a stochastic policy for generating com… (see more)plex combinatorial structure through a series of decision-making steps. Despite being inspired from reinforcement learning, the current GFlowNet framework is relatively limited in its applicability and cannot handle stochasticity in the reward function. In this work, we adopt a distributional paradigm for GFlowNets, turning each flow function into a distribution, thus providing more informative learning signals during training. By parameterizing each edge flow through their quantile functions, our proposed \textit{quantile matching} GFlowNet learning algorithm is able to learn a risk-sensitive policy, an essential component for handling scenarios with risk uncertainty. Moreover, we find that the distributional approach can achieve substantial improvement on existing benchmarks compared to prior methods due to our enhanced training algorithm, even in settings with deterministic rewards.

2024-02-15

Transactions on Machine Learning Research (accepted)

Computing Power and the Governance of Artificial Intelligence

Girish Sastry

Lennart Heim

Haydn Belfield

Markus Anderljung

Miles Brundage

Julian Hazell

Cullen C. O'keefe

Gillian K. Hadfield

Richard Ngo

Konstantin Pilz

George Gor

Emma Bluemke

Sarah Shoker

Janet Egan

Robert Trager

Shahar Avin

Adrian Weller

Diane Coyle

Computing power, or"compute,"is crucial for the development and deployment of artificial intelligence (AI) capabilities. As a result, govern… (see more)ments and companies have started to leverage compute as a means to govern AI. For example, governments are investing in domestic compute capacity, controlling the flow of compute to competing countries, and subsidizing compute access to certain sectors. However, these efforts only scratch the surface of how compute can be used to govern AI development and deployment. Relative to other key inputs to AI (data and algorithms), AI-relevant compute is a particularly effective point of intervention: it is detectable, excludable, and quantifiable, and is produced via an extremely concentrated supply chain. These characteristics, alongside the singular importance of compute for cutting-edge AI models, suggest that governing compute can contribute to achieving common policy objectives, such as ensuring the safety and beneficial use of AI. More precisely, policymakers could use compute to facilitate regulatory visibility of AI, allocate resources to promote beneficial outcomes, and enforce restrictions against irresponsible or malicious AI development and usage. However, while compute-based policies and technologies have the potential to assist in these areas, there is significant variation in their readiness for implementation. Some ideas are currently being piloted, while others are hindered by the need for fundamental research. Furthermore, naive or poorly scoped approaches to compute governance carry significant risks in areas like privacy, economic impacts, and centralization of power. We end by suggesting guardrails to minimize these risks from compute governance.

2024-02-12

ArXiv (preprint)

Consciousness-Inspired Spatio-Temporal Abstractions for Better Generalization in Reinforcement Learning

Mingde Zhao

Harm van Seijen

Romain Laroche

Inspired by human conscious planning, we propose Skipper, a model-based reinforcement learning framework utilizing spatio-temporal abstracti… (see more)ons to generalize better in novel situations. It automatically decomposes the given task into smaller, more manageable subtasks, and thus enables sparse decision-making and focused computation on the relevant parts of the environment. The decomposition relies on the extraction of an abstracted proxy problem represented as a directed graph, in which vertices and edges are learned end-to-end from hindsight. Our theoretical analyses provide performance guarantees under appropriate assumptions and establish where our approach is expected to be helpful. Generalization-focused experiments validate Skipper's significant advantage in zero-shot generalization, compared to some existing state-of-the-art hierarchical planning methods.

2024-01-15

International Conference on Learning Representations (poster)

blocks: Version 0.2.0

Bart Van Merriënboer

Dzmitry Bahdanau

Dzmitry Bahdanau

Vincent Dumoulin

David Warde-Farley

A Theano framework for building and training neural networks

2023-12-31

International Conference on Machine Learning (published)

Generative Active Learning for the Search of Small-Molecule Protein Binders

Maksym Korablyov

Cheng-Hao Liu

Almer Van Der Sloot

Éric Jolicoeur

Edward Ruediger

Andrei Nica

Emmanuel Bengio

Kostiantyn Lapchevskyi

Daniel St-Cyr

Doris Alexandra Schuetz

Victor Ion Butoi

Jarrid Rector-Brooks

Simon Blackburn

Saikrishna Gottipati

Priyesh Vijayan

Prateek Gupta

Ladislav Rampasek … (see 14 more)

Sasikanth Avancha

Pierre-Luc Bacon

William Hamilton

Brooks Paige

Sanchit Misra

Stanislaw Jastrzebski

Bharat Kaul

José Miguel Hernández-Lobato

Marwin Segler

Michael Bronstein

Anne Marinier

Mike Tyers

Despite substantial progress in machine learning for scientific discovery in recent years, truly de novo design of small molecules which exh… (see more)ibit a property of interest remains a significant challenge. We introduce LambdaZero, a generative active learning approach to search for synthesizable molecules. Powered by deep reinforcement learning, LambdaZero learns to search over the vast space of molecules to discover candidates with a desired property. We apply LambdaZero with molecular docking to design novel small molecules that inhibit the enzyme soluble Epoxide Hydrolase 2 (sEH), while enforcing constraints on synthesizability and drug-likeliness. LambdaZero provides an exponential speedup in terms of the number of calls to the expensive molecular docking oracle, and LambdaZero de novo designed molecules reach docking scores that would otherwise require the virtual screening of a hundred billion molecules. Importantly, LambdaZero discovers novel scaffolds of synthesizable, drug-like inhibitors for sEH. In in vitro experimental validation, a series of ligands from a generated quinazoline-based scaffold were synthesized, and the lead inhibitor N-(4,6-di(pyrrolidin-1-yl)quinazolin-2-yl)-N-methylbenzamide (UM0152893) displayed sub-micromolar enzyme inhibition of sEH.

2023-12-31

arXiv (preprint)

Speech Model Pre-training for End-to-End Spoken Language Understanding

Mirco Ravanelli

Patrick Ignoto

Vikrant Singh Tomar

Whereas conventional spoken language understanding (SLU) systems map speech to text, and then text to intent, end-to-end SLU systems map spe… (see more)ech directly to intent through a single trainable model. Achieving high accuracy with these end-to-end models without a large amount of training data is difficult. We propose a method to reduce the data requirements of end-to-end SLU in which the model is first pre-trained to predict words and phonemes, thus learning good features for SLU. We introduce a new SLU dataset, Fluent Speech Commands, and show that our method improves performance both when the full dataset is used for training and when only a small subset is used. We also describe preliminary experiments to gauge the model's ability to generalize to new phrases not heard during training.

2023-12-31

TIB (published)