Portrait of Yoshua Bengio

Yoshua Bengio

Core Academic Member
Canada CIFAR AI Chair
Full Professor, Université de Montréal, Department of Computer Science and Operations Research Department
Founder and Scientific Advisor, Leadership Team
Research Topics
Causality
Computational Neuroscience
Deep Learning
Generative Models
Graph Neural Networks
Machine Learning Theory
Medical Machine Learning
Molecular Modeling
Natural Language Processing
Probabilistic Models
Reasoning
Recurrent Neural Networks
Reinforcement Learning
Representation Learning
Website
Google Scholar

Biography

*For media requests, please write to medias@mila.quebec.

For more information please contact Marie-Josée Beauchamp, Administrative Assistant at marie-josee.beauchamp@mila.quebec.

Yoshua Bengio is recognized worldwide as a leading expert in AI. He is most known for his pioneering work in deep learning, which earned him the 2018 A.M. Turing Award, “the Nobel Prize of computing,” with Geoffrey Hinton and Yann LeCun.

Bengio is a full professor at Université de Montréal, and the founder and scientific advisor of Mila – Quebec Artificial Intelligence Institute. He is also a senior fellow at CIFAR and co-directs its Learning in Machines & Brains program, serves as special advisor and founding scientific director of IVADO, and holds a Canada CIFAR AI Chair.

In 2019, Bengio was awarded the prestigious Killam Prize and in 2022, he was the most cited computer scientist in the world by h-index. He is a Fellow of the Royal Society of London, Fellow of the Royal Society of Canada, Knight of the Legion of Honor of France and Officer of the Order of Canada. In 2023, he was appointed to the UN’s Scientific Advisory Board for Independent Advice on Breakthroughs in Science and Technology.

Concerned about the social impact of AI, Bengio helped draft the Montréal Declaration for the Responsible Development of Artificial Intelligence and continues to raise awareness about the importance of mitigating the potentially catastrophic risks associated with future AI systems.

Current Students

Jamal Abou Haibeh
Collaborating Alumni - McGill University
Github
Google Scholar
Mohammed Abukalam
Collaborating Alumni - Université de Montréal
Github
Berkes Anaïs
Collaborating researcher - Cambridge University
Principal supervisor :
David Rolnick
Website
Github
Google Scholar
Rim Assouel
PhD - Université de Montréal
Stefan Bauer
Independent visiting researcher
Co-supervisor :
Guillaume Lajoie
Google Scholar
Paul Bertin
PhD - Université de Montréal
Joyce Chai
Independent visiting researcher
Principal supervisor :
Siva Reddy
Shahana Chatterjee
Collaborating researcher - N/A
Principal supervisor :
David Rolnick
Google Scholar
Xiaoyin Chen
PhD - Université de Montréal
Sanghyeok Choi
Collaborating researcher - KAIST
Website
Github
Google Scholar
Tristan Deleu
Collaborating Alumni - Université de Montréal
Website
Github
Google Scholar
Aniket Didolkar
PhD - Université de Montréal
Website
Github
Google Scholar
Abdessamad EL KABID
Collaborating Alumni - Université de Montréal
Co-supervisor :
Loubna Benabbou
Desmond Elliott
Independent visiting researcher
Principal supervisor :
Siva Reddy
Website
Google Scholar
Eric Elmoznino
PhD - Université de Montréal
Co-supervisor :
Guillaume Lajoie
Website
Github
Google Scholar
Léna Ezzine
PhD - Université de Montréal
Jean-Pierre Falet
PhD - Université de Montréal
Website
Github
Google Scholar
Leo Feng
PhD - Université de Montréal
Website
Google Scholar
Pietro Greiner
PhD
Google Scholar
Mohsin Hasan
PhD - Université de Montréal
Website
Github
Google Scholar
Edward Hu
PhD - Université de Montréal
Moksh Jain
PhD - Université de Montréal
Website
Github
Google Scholar
Thomas Jiralerspong
PhD - Université de Montréal
Principal supervisor :
Guillaume Lajoie
Website
Github
Google Scholar
Younesse Kaddar
Collaborating Alumni - Université de Montréal
Website
Github
Hyeonah Kim
Postdoctorate - Université de Montréal
Principal supervisor :
Alex Hernandez-Garcia
Salem Lahlou
Collaborating Alumni - Université de Montréal
Tabitha Edith Lee
Postdoctorate - Université de Montréal
Principal supervisor :
Glen Berseth
Website
Github
Google Scholar
Chenghao Liu
Collaborating Alumni
Website
Github
Google Scholar
Zhen Liu
Collaborating Alumni - Université de Montréal
Principal supervisor :
Liam Paull
Kanika Madan
PhD - Université de Montréal
Nikolay Malkin
Collaborating Alumni - Université de Montréal
Website
Github
Google Scholar
Cristian Dragos Manta
PhD - Université de Montréal
Co-supervisor :
Dhanya Sridhar
Github
Sarthak Mittal
PhD - Université de Montréal
Principal supervisor :
Guillaume Lajoie
Website
Github
Google Scholar
Jama Mohamud
PhD - Université de Montréal
Principal supervisor :
Mirco Ravanelli
Website
Github
Google Scholar
Cyrus Neary
Postdoctorate - Université de Montréal
Principal supervisor :
Glen Berseth
Website
Github
Google Scholar
Phong Nguyen
Independent visiting researcher - Université de Montréal
Padideh Nouri
PhD - Université de Montréal
Principal supervisor :
Doina Precup
Website
Github
Google Scholar
Ali Parviz
Collaborating researcher - Ying Wu Coll of Computing
Google Scholar
Lena Podina
Collaborating researcher - University of Waterloo
Principal supervisor :
David Rolnick
Google Scholar
Nassim Rahaman
Collaborating Alumni - Max-Planck-Institute for Intelligent Systems
Amine RAZIG
Collaborating researcher - Université de Montréal
Co-supervisor :
Loubna Benabbou
Website
Jarrid Rector-Brooks
PhD - Université de Montréal
Danyal REHMAN
Postdoctorate - Université de Montréal
Github
James Requeima
Independent visiting researcher - Université de Montréal
Oli RICHARDSON
Postdoctorate - Université de Montréal
Website
Google Scholar
Camille Rochefort-Boulanger
PhD - Université de Montréal
Principal supervisor :
Julie Hussin
Github
Abhik Roychoudhury Roychoudhury
Independent visiting researcher
Principal supervisor :
Siva Reddy
Github
Luca Scimeca
Postdoctorate - Université de Montréal
Website
Github
Google Scholar
Dragos Secrieru
Collaborating Alumni - Université de Montréal
Marcin Sendera
Collaborating Alumni - Université de Montréal
Github
Google Scholar
Divya Sharma
Postdoctorate
Co-supervisor :
Alex Hernandez-Garcia
Website
Github
Marco Stock
Independent visiting researcher - Technical University of Munich
Github
Mélisande Astrid Crystal Teng
PhD - Université de Montréal
Co-supervisor :
Hugo Larochelle
Website
Github
Ivan Titov
Independent visiting researcher
Principal supervisor :
Siva Reddy
Website
Google Scholar
Alex Tong
Collaborating Alumni - Université de Montréal
Website
Github
Google Scholar
Donna Vakalis
Postdoctorate - Université de Montréal
Co-supervisor :
David Rolnick
Website
Github
Google Scholar
Siddarth Venkatraman
PhD - Université de Montréal
Principal supervisor :
Glen Berseth
Website
Github
Google Scholar
Omar G. Younis
Collaborating researcher
Website
Github
Google Scholar
Taeyoung YUN
Collaborating researcher - Université de Montréal
Github
Tianyu Zhang
PhD - Université de Montréal
Website
Github
Google Scholar
Nicole Zhang
PhD - McGill University
Principal supervisor :
Mathieu Blanchette
Website
Github
Google Scholar
Dinghuai Zhang
PhD - Université de Montréal
Principal supervisor :
Aaron Courville
Website
Harry Zhao
Collaborating Alumni - McGill University
Principal supervisor :
Doina Precup
Website
Github
Google Scholar

Publications

VCR: A Task for Pixel-Level Complex Reasoning in Vision Language Models via Restoring Occluded Text
Tianyu Zhang
Suyuchen Wang
Li Li
Ge Zhang
Perouz Taslakian
Sai Rajeswar
Jie Fu
Bang Liu
Yoshua Bengio
We introduce Visual Caption Restoration (VCR), a novel vision-language task that challenges models to accurately restore partially obscured … (see more)texts using pixel-level hints within images. This task stems from the observation that text embedded in images is intrinsically different from common visual elements and natural language due to the need to align the modalities of vision, text, and text embedded in images. While numerous works have integrated text embedded in images into visual question-answering tasks, approaches to these tasks generally rely on optical character recognition or masked language modeling, thus reducing the task to mainly text-based processing. However, text-based processing becomes ineffective in VCR as accurate text restoration depends on the combined information from provided images, context, and subtle cues from the tiny exposed areas of masked texts. We develop a pipeline to generate synthetic images for the VCR task using image-caption pairs, with adjustable caption visibility to control the task difficulty. With this pipeline, we construct a dataset for VCR called VCR-Wiki using images with captions from Wikipedia, comprising 2.11M English and 346K Chinese entities in both easy and hard split variants. Our results reveal that current vision language models significantly lag behind human performance in the VCR task, and merely fine-tuning the models on our dataset does not lead to notable improvements. We release VCR-Wiki and the data construction code to facilitate future research.
2024-06-10
ArXiv (preprint)
arxiv.org
arxiv.org
VCR: A Task for Pixel-Level Complex Reasoning in Vision Language Models via Restoring Occluded Text
Tianyu Zhang
Suyuchen Wang
Li Li
Ge Zhang
Perouz Taslakian
Sai Rajeswar
Jie Fu
Bang Liu
Yoshua Bengio
We introduce Visual Caption Restoration (VCR), a novel vision-language task that challenges models to accurately restore partially obscured … (see more)texts using pixel-level hints within images. This task stems from the observation that text embedded in images is intrinsically different from common visual elements and natural language due to the need to align the modalities of vision, text, and text embedded in images. While numerous works have integrated text embedded in images into visual question-answering tasks, approaches to these tasks generally rely on optical character recognition or masked language modeling, thus reducing the task to mainly text-based processing. However, text-based processing becomes ineffective in VCR as accurate text restoration depends on the combined information from provided images, context, and subtle cues from the tiny exposed areas of masked texts. We develop a pipeline to generate synthetic images for the VCR task using image-caption pairs, with adjustable caption visibility to control the task difficulty. With this pipeline, we construct a dataset for VCR called VCR-Wiki using images with captions from Wikipedia, comprising 2.11M English and 346K Chinese entities in both easy and hard split variants. Our results reveal that current vision language models significantly lag behind human performance in the VCR task, and merely fine-tuning the models on our dataset does not lead to notable improvements. We release VCR-Wiki and the data construction code to facilitate future research.
2024-06-10
ArXiv (preprint)
arxiv.org
arxiv.org
VCR: Visual Caption Restoration
Tianyu Zhang
Suyuchen Wang
Lu Li
Ge Zhang
Perouz Taslakian
Sai Rajeswar
Jie Fu
Bang Liu
Yoshua Bengio
We introduce Visual Caption Restoration (VCR), a novel vision-language task that challenges models to accurately restore partially obscured … (see more)texts using pixel-level hints within images. This task stems from the observation that text embedded in images is intrinsically different from common visual elements and natural language due to the need to align the modalities of vision, text, and text embedded in images. While numerous works have integrated text embedded in images into visual question-answering tasks, approaches to these tasks generally rely on optical character recognition or masked language modeling, thus reducing the task to mainly text-based processing. However, text-based processing becomes ineffective in VCR as accurate text restoration depends on the combined information from provided images, context, and subtle cues from the tiny exposed areas of masked texts. We develop a pipeline to generate synthetic images for the VCR task using image-caption pairs, with adjustable caption visibility to control the task difficulty. With this pipeline, we construct a dataset for VCR called VCR-Wiki using images with captions from Wikipedia, comprising 2.11M English and 346K Chinese entities in both easy and hard split variants. Our results reveal that current vision language models significantly lag behind human performance in the VCR task, and merely fine-tuning the models on our dataset does not lead to notable improvements. We release VCR-Wiki and the data construction code to facilitate future research.
2024-06-10
ArXiv (preprint)
doi.org
arxiv.org
VCR: Visual Caption Restoration
Tianyu Zhang
Suyuchen Wang
Lu Li
Ge Zhang
Perouz Taslakian
Sai Rajeswar
Jie Fu
Bang Liu
Yoshua Bengio
We introduce Visual Caption Restoration (VCR), a novel vision-language task that challenges models to accurately restore partially obscured … (see more)texts using pixel-level hints within images. This task stems from the observation that text embedded in images is intrinsically different from common visual elements and natural language due to the need to align the modalities of vision, text, and text embedded in images. While numerous works have integrated text embedded in images into visual question-answering tasks, approaches to these tasks generally rely on optical character recognition or masked language modeling, thus reducing the task to mainly text-based processing. However, text-based processing becomes ineffective in VCR as accurate text restoration depends on the combined information from provided images, context, and subtle cues from the tiny exposed areas of masked texts. We develop a pipeline to generate synthetic images for the VCR task using image-caption pairs, with adjustable caption visibility to control the task difficulty. With this pipeline, we construct a dataset for VCR called VCR-Wiki using images with captions from Wikipedia, comprising 2.11M English and 346K Chinese entities in both easy and hard split variants. Our results reveal that current vision language models significantly lag behind human performance in the VCR task, and merely fine-tuning the models on our dataset does not lead to notable improvements. We release VCR-Wiki and the data construction code to facilitate future research.
2024-06-10
ArXiv (preprint)
doi.org
arxiv.org
VCR: Visual Caption Restoration
Tianyu Zhang
Suyuchen Wang
Lu Li
Ge Zhang
Perouz Taslakian
Sai Rajeswar
Jie Fu
Bang Liu
Yoshua Bengio
We introduce Visual Caption Restoration (VCR), a novel vision-language task that challenges models to accurately restore partially obscured … (see more)texts using pixel-level hints within images. This task stems from the observation that text embedded in images is intrinsically different from common visual elements and natural language due to the need to align the modalities of vision, text, and text embedded in images. While numerous works have integrated text embedded in images into visual question-answering tasks, approaches to these tasks generally rely on optical character recognition or masked language modeling, thus reducing the task to mainly text-based processing. However, text-based processing becomes ineffective in VCR as accurate text restoration depends on the combined information from provided images, context, and subtle cues from the tiny exposed areas of masked texts. We develop a pipeline to generate synthetic images for the VCR task using image-caption pairs, with adjustable caption visibility to control the task difficulty. With this pipeline, we construct a dataset for VCR called VCR-Wiki using images with captions from Wikipedia, comprising 2.11M English and 346K Chinese entities in both easy and hard split variants. Our results reveal that current vision language models significantly lag behind human performance in the VCR task, and merely fine-tuning the models on our dataset does not lead to notable improvements. We release VCR-Wiki and the data construction code to facilitate future research.
2024-06-10
ArXiv (preprint)
doi.org
arxiv.org
Baking Symmetry into GFlowNets
George Ma
Emmanuel Bengio
Yoshua Bengio
Dinghuai Zhang
GFlowNets have exhibited promising performance in generating diverse candidates with high rewards. These networks generate objects increment… (see more)ally and aim to learn a policy that assigns probability of sampling objects in proportion to rewards. However, the current training pipelines of GFlowNets do not consider the presence of isomorphic actions, which are actions resulting in symmetric or isomorphic states. This lack of symmetry increases the amount of samples required for training GFlowNets and can result in inefficient and potentially incorrect flow functions. As a consequence, the reward and diversity of the generated objects decrease. In this study, our objective is to integrate symmetries into GFlowNets by identifying equivalent actions during the generation process. Experimental results using synthetic data demonstrate the promising performance of our proposed approaches.
2024-06-08
ArXiv (preprint)
doi.org
arxiv.org
RGFN: Synthesizable Molecular Generation Using GFlowNets
Michał Koziarski
Andrei Rekesh
Dmytro Shevchuk
Almer M. van der Sloot
Piotr Gainski
Yoshua Bengio
Cheng-Hao Liu
Mike Tyers
Robert A. Batey
Generative models hold great promise for small molecule discovery, significantly increasing the size of search space compared to traditional… (see more) in silico screening libraries. However, most existing machine learning methods for small molecule generation suffer from poor synthesizability of candidate compounds, making experimental validation difficult. In this paper we propose Reaction-GFlowNet (RGFN), an extension of the GFlowNet framework that operates directly in the space of chemical reactions, thereby allowing out-of-the-box synthesizability while maintaining comparable quality of generated candidates. We demonstrate that with the proposed set of reactions and building blocks, it is possible to obtain a search space of molecules orders of magnitude larger than existing screening libraries coupled with low cost of synthesis. We also show that the approach scales to very large fragment libraries, further increasing the number of potential molecules. We demonstrate the effectiveness of the proposed approach across a range of oracle models, including pretrained proxy models and GPU-accelerated docking.
2024-06-01
ArXiv (preprint)
doi.org
arxiv.org
RGFN: Synthesizable Molecular Generation Using GFlowNets
Michał Koziarski
Andrei Rekesh
Dmytro Shevchuk
Almer M. van der Sloot
Piotr Gainski
Yoshua Bengio
Cheng-Hao Liu
Mike Tyers
Robert A. Batey
Generative models hold great promise for small molecule discovery, significantly increasing the size of search space compared to traditional… (see more) in silico screening libraries. However, most existing machine learning methods for small molecule generation suffer from poor synthesizability of candidate compounds, making experimental validation difficult. In this paper we propose Reaction-GFlowNet (RGFN), an extension of the GFlowNet framework that operates directly in the space of chemical reactions, thereby allowing out-of-the-box synthesizability while maintaining comparable quality of generated candidates. We demonstrate that with the proposed set of reactions and building blocks, it is possible to obtain a search space of molecules orders of magnitude larger than existing screening libraries coupled with low cost of synthesis. We also show that the approach scales to very large fragment libraries, further increasing the number of potential molecules. We demonstrate the effectiveness of the proposed approach across a range of oracle models, including pretrained proxy models and GPU-accelerated docking.
2024-06-01
ArXiv (preprint)
doi.org
arxiv.org
RGFN: Synthesizable Molecular Generation Using GFlowNets
Michał Koziarski
Andrei Rekesh
Dmytro Shevchuk
Almer M. van der Sloot
Piotr Gainski
Yoshua Bengio
Cheng-Hao Liu
Mike Tyers
Robert A. Batey
Generative models hold great promise for small molecule discovery, significantly increasing the size of search space compared to traditional… (see more) in silico screening libraries. However, most existing machine learning methods for small molecule generation suffer from poor synthesizability of candidate compounds, making experimental validation difficult. In this paper we propose Reaction-GFlowNet (RGFN), an extension of the GFlowNet framework that operates directly in the space of chemical reactions, thereby allowing out-of-the-box synthesizability while maintaining comparable quality of generated candidates. We demonstrate that with the proposed set of reactions and building blocks, it is possible to obtain a search space of molecules orders of magnitude larger than existing screening libraries coupled with low cost of synthesis. We also show that the approach scales to very large fragment libraries, further increasing the number of potential molecules. We demonstrate the effectiveness of the proposed approach across a range of oracle models, including pretrained proxy models and GPU-accelerated docking.
2024-06-01
ArXiv (preprint)
doi.org
arxiv.org
Amortizing intractable inference in diffusion models for vision, language, and control
Siddarth Venkatraman
Moksh J. Jain
Luca Scimeca
Minsu Kim
Marcin Sendera
Mohsin Hasan
Luke Rowe
Sarthak Mittal
Pablo Lemos
Emmanuel Bengio
Alexandre Adam
Jarrid Rector-Brooks
Yoshua Bengio
Glen Berseth
Nikolay Malkin
Diffusion models have emerged as effective distribution estimators in vision, language, and reinforcement learning, but their use as priors … (see more)in downstream tasks poses an intractable posterior inference problem. This paper studies amortized sampling of the posterior over data,
2024-05-31
ArXiv (preprint)
doi.org
arxiv.org
Amortizing intractable inference in diffusion models for vision, language, and control
Siddarth Venkatraman
Moksh J. Jain
Luca Scimeca
Minsu Kim
Marcin Sendera
Mohsin Hasan
Luke Rowe
Sarthak Mittal
Pablo Lemos
Emmanuel Bengio
Alexandre Adam
Jarrid Rector-Brooks
Yoshua Bengio
Glen Berseth
Nikolay Malkin
Diffusion models have emerged as effective distribution estimators in vision, language, and reinforcement learning, but their use as priors … (see more)in downstream tasks poses an intractable posterior inference problem. This paper studies amortized sampling of the posterior over data,
2024-05-31
ArXiv (preprint)
doi.org
arxiv.org
Amortizing intractable inference in diffusion models for vision, language, and control
Siddarth Venkatraman
Moksh J. Jain
Luca Scimeca
Minsu Kim
Marcin Sendera
Mohsin Hasan
Luke Rowe
Sarthak Mittal
Pablo Lemos
Emmanuel Bengio
Alexandre Adam
Jarrid Rector-Brooks
Yoshua Bengio
Glen Berseth
Nikolay Malkin
Diffusion models have emerged as effective distribution estimators in vision, language, and reinforcement learning, but their use as priors … (see more)in downstream tasks poses an intractable posterior inference problem. This paper studies amortized sampling of the posterior over data,
2024-05-31
ArXiv (preprint)
doi.org
arxiv.org