Portrait de Yoshua Bengio

Yoshua Bengio

Membre académique principal

Chaire en IA Canada-CIFAR

Professeur titulaire, Université de Montréal, Département d'informatique et de recherche opérationnelle

Sujets de recherche

Apprentissage automatique médical

Apprentissage de représentations

Apprentissage par renforcement

Apprentissage profond

Causalité

Modèles génératifs

Modèles probabilistes

Modélisation moléculaire

Neurosciences computationnelles

Raisonnement

Réseaux de neurones en graphes

Réseaux de neurones récurrents

Théorie de l'apprentissage automatique

Traitement du langage naturel

Biographie

*Pour toute demande média, veuillez écrire à medias@mila.quebec.

Pour plus d’information, contactez Cassidy MacNeil, adjointe principale et responsable des opérations cassidy.macneil@mila.quebec.

Reconnu comme une sommité mondiale en intelligence artificielle, Yoshua Bengio s’est surtout distingué par son rôle de pionnier en apprentissage profond, ce qui lui a valu le prix A. M. Turing 2018, le « prix Nobel de l’informatique », avec Geoffrey Hinton et Yann LeCun. Il est professeur titulaire à l’Université de Montréal, fondateur et conseiller scientifique de Mila – Institut québécois d’intelligence artificielle, et codirige en tant que senior fellow le programme Apprentissage automatique, apprentissage biologique de l'Institut canadien de recherches avancées (CIFAR). Il occupe également la fonction de conseiller spécial et directeur scientifique fondateur d’IVADO.

En 2018, il a été l’informaticien qui a recueilli le plus grand nombre de nouvelles citations au monde. En 2019, il s’est vu décerner le prestigieux prix Killam. Depuis 2022, il détient le plus grand facteur d’impact (h-index) en informatique à l’échelle mondiale. Il est fellow de la Royal Society de Londres et de la Société royale du Canada, et officier de l’Ordre du Canada.

Soucieux des répercussions sociales de l’IA et de l’objectif que l’IA bénéficie à tous, il a contribué activement à la Déclaration de Montréal pour un développement responsable de l’intelligence artificielle.

Étudiants actuels

Jamal Abou Haibeh

Collaborateur·rice alumni - McGill

jamal.abouhaibeh@mila.quebec

Collaborateur·rice de recherche - Cambridge University

Superviseur⋅e principal⋅e :

berkes.anais@mila.quebec

Doctorat - UdeM

assouelr@mila.quebec

Shahana Chatterjee

Collaborateur·rice de recherche - N/A

Superviseur⋅e principal⋅e :

shahana.chatterjee@mila.quebec

Doctorat - UdeM

xiaoyin.chen@mila.quebec

Collaborateur·rice de recherche - KAIST

sanghyeok.choi@mila.quebec

Aniket Didolkar

Doctorat - UdeM

aniket.didolkar@mila.quebec

Doctorat - UdeM

Co-superviseur⋅e :

Guillaume Lajoie

eric.elmoznino@mila.quebec

Doctorat - UdeM

lena-nehale.ezzine@mila.quebec

Jean-Pierre Falet

Doctorat - UdeM

jean-pierre.falet@mila.quebec

Doctorat

pietro.greiner@mila.quebec

Doctorat - UdeM

mohsin.hasan@mila.quebec

Doctorat - UdeM

moksh.jain@mila.quebec

Thomas Jiralerspong

Doctorat - UdeM

Superviseur⋅e principal⋅e :

Guillaume Lajoie

thomas.jiralerspong@mila.quebec

Younesse Kaddar

Collaborateur·rice alumni - UdeM

younesse.kaddar@mila.quebec

Collaborateur·rice de recherche - UdeM

minsu.kim@mila.quebec

Postdoctorat - UdeM

Superviseur⋅e principal⋅e :

Alex Hernández-García

hyeonah.kim@mila.quebec

Stagiaire de recherche - KOREA ADVANCED INSTITUTE OF SCIENCE AND TECHNOLOGY

jeongjae.lee@mila.quebec

Collaborateur·rice de recherche - s.o.

liusong299@gmail.com

Collaborateur·rice de recherche

liucheng@mila.quebec

Cristian Dragos Manta

Doctorat - UdeM

Co-superviseur⋅e :

cristian-dragos.manta@mila.quebec

Doctorat - UdeM

Superviseur⋅e principal⋅e :

Guillaume Lajoie

mittalsa@mila.quebec

Visiteur de recherche indépendant - UdeM

nguyenph@mila.quebec

Collaborateur·rice de recherche - Ying Wu Coll of Computing

ali.parviz@mila.quebec

Collaborateur·rice de recherche - University of Waterloo

Superviseur⋅e principal⋅e :

elena.podina@mila.quebec

Jarrid Rector-Brooks

Doctorat - UdeM

jarrid.rector-brooks@mila.quebec

Postdoctorat - UdeM

danyal.rehman@mila.quebec

Postdoctorat - UdeM

oliver.richardson@mila.quebec

Camille Rochefort-Boulanger

Doctorat - UdeM

Superviseur⋅e principal⋅e :

rochefoc@mila.quebec

Postdoctorat - UdeM

Co-superviseur⋅e :

Alex Hernández-García

divya.sharma@mila.quebec

Mélisande Astrid Crystal Teng

Collaborateur·rice alumni - UdeM

Co-superviseur⋅e :

Hugo Larochelle

tengmeli@mila.quebec

Collaborateur·rice alumni - UdeM

alexander.tong@mila.quebec

Collaborateur·rice de recherche - University of Cambridge

oszkar.urban@mila.quebec

Collaborateur·rice alumni - UdeM

Co-superviseur⋅e :

donna.vakalis@mila.quebec

Siddarth Venkatraman

Doctorat - UdeM

Superviseur⋅e principal⋅e :

siddarth.venkatraman@mila.quebec

Esmeralda Whitammer

Collaborateur·rice de recherche - UdeM

esmeralda.whitammer@mila.quebec

Collaborateur·rice alumni

omar.younis@mila.quebec

Collaborateur·rice de recherche - UdeM

taeyoung.yun@mila.quebec

Doctorat - UdeM

tianyu.zhang@mila.quebec

Doctorat - McGill

Superviseur⋅e principal⋅e :

Mathieu Blanchette

xi.zhang@mila.quebec

Billets de blogue

Generic thumbnail for Mila Blog articles.

22 février 2024

Skipper : combiner l’abstraction spatiale et temporelle afin d’améliorer la généralisation

par

Mingde Harry Zhao

Safa Alver

Harm van Seijen

Romain Laroche

Scaling in the service of reasoning & model-based ML

4 avril 2023

Mise à l’échelle au service du raisonnement et de l’apprentissage automatique basé sur un modèle

par

A collaboration between Mila and Relation Therapeutics to discover novel synergistic combinations of drugs in vitro

23 mars 2022

Une collaboration entre Mila et Relation Therapeutics pour découvrir in vitro de nouvelles associations médicamenteuses synergiques

par

Jake P. Taylor-King

Generative Flow Networks

15 mars 2022

Les réseaux de flot génératifs

par

Publications

AI for Global Climate Cooperation: Modeling Global Climate Negotiations, Agreements, and Long-Term Cooperation in RICE-N

Andrew Williams

Phillip Wozny

Kai-Hendrik Cohrs

Koen Ponse

Marco Jiralerspong

Soham Phade

Sunil Srinivasa

Yang Zhang

Prateek Gupta

Erman Acar

Stephan Zheng

Comprehensive global cooperation is essential to limit global temperature increases while continuing economic development, e.g., reducing se… (voir plus)vere inequality or achieving long-term economic growth. Achieving long-term cooperation on climate change mitigation with n strategic agents poses a complex game-theoretic problem. For example, agents may negotiate and reach climate agreements, but there is no central authority to enforce adherence to those agreements. Hence, it is critical to design negotiation and agreement frameworks that foster cooperation, allow all agents to meet their individual policy objectives, and incentivize long-term adherence. This is an interdisciplinary challenge that calls for collaboration between researchers in machine learning, economics, climate science, law, policy, ethics, and other fields. In particular, we argue that machine learning is a critical tool to address the complexity of this domain. To facilitate this research, here we introduce RICE-N, a multi-region integrated assessment model that simulates the global climate and economy, and which can be used to design and evaluate the strategic outcomes for different negotiation and agreement frameworks. We also describe how to use multi-agent reinforcement learning to train rational agents using RICE-N. This framework underpinsAI for Global Climate Cooperation, a working group collaboration and competition on climate negotiation and agreement design. Here, we invite the scientific community to design and evaluate their solutions using RICE-N, machine learning, economic intuition, and other domain knowledge. More information can be found on www.ai4climatecoop.org.

2025-04-30

International Conference on Machine Learning (poster)

proceedings.mlr.press

Outsourced Diffusion Sampling: Efficient Posterior Inference in Latent Spaces of Generative Models

Siddarth Venkatraman

Nikolay Malkin

Any well-behaved generative model over a variable …

2025-04-30

International Conference on Machine Learning (poster)

proceedings.mlr.press

Adaptive Teachers for Amortized Samplers

Emmanuel Bengio

Jarrid Rector-Brooks

Sungsoo Ahn

Jinkyoo Park

Nikolay Malkin

Amortized inference is the task of training a parametric model, such as a neural network, to approximate a distribution with a given unnorma… (voir plus)lized density where exact sampling is intractable. When sampling is implemented as a sequential decision-making process, reinforcement learning (RL) methods, such as generative flow networks, can be used to train the sampling policy. Off-policy RL training facilitates the discovery of diverse, high-reward candidates, but existing methods still face challenges in efficient exploration. We propose to use an adaptive training distribution (the \teacher) to guide the training of the primary amortized sampler (the \student). The \teacher, an auxiliary behavior model, is trained to sample high-loss regions of the \student and can generalize across unexplored modes, thereby enhancing mode coverage by providing an efficient training curriculum. We validate the effectiveness of this approach in a synthetic environment designed to present an exploration challenge, two diffusion-based sampling tasks, and four biochemical discovery tasks demonstrating its ability to improve sample efficiency and mode coverage. Source code is available at https://github.com/alstn12088/adaptive-teacher.

2025-04-22

International Conference on Learning Representations (Accept (Poster))

AssembleFlow: Rigid Flow Matching with Inertial Frames for Molecular Assembly

Hongyu Guo

2025-04-22

International Conference on Learning Representations (Accept (Poster))

Efficient Diversity-Preserving Diffusion Alignment via Gradient-Informed GFlowNets

Tim Z. Xiao

Weiyang Liu

While one commonly trains large diffusion models by collecting datasets on target downstream tasks, it is often desired to align and finetun… (voir plus)e pretrained diffusion models with some reward functions that are either designed by experts or learned from small-scale datasets. Existing post-training methods for reward finetuning of diffusion models typically suffer from lack of diversity in generated samples, lack of prior preservation, and/or slow convergence in finetuning. In response to this challenge, we take inspiration from recent successes in generative flow networks (GFlowNets) and propose a reinforcement learning method for diffusion model finetuning, dubbed Nabla-GFlowNet (abbreviated as

2025-04-22

International Conference on Learning Representations (Accept (Poster))

HarmAug: Effective Data Augmentation for Knowledge Distillation of Safety Guard Models

Haebin Seong

Dong Bok Lee

Minki Kang

Dominik Wagner

Juho Lee

Sung Ju Hwang

Safety guard models that detect malicious queries aimed at large language models (LLMs) are essential for ensuring the secure and responsibl… (voir plus)e deployment of LLMs in real-world applications. However, deploying existing safety guard models with billions of parameters alongside LLMs on mobile devices is impractical due to substantial memory requirements and latency. To reduce this cost, we distill a large teacher safety guard model into a smaller one using a labeled dataset of instruction-response pairs with binary harmfulness labels. Due to the limited diversity of harmful instructions in the existing labeled dataset, naively distilled models tend to underperform compared to larger models. To bridge the gap between small and large models, we propose HarmAug, a simple yet effective data augmentation method that involves jailbreaking an LLM and prompting it to generate harmful instructions. Given a prompt such as, "Make a single harmful instruction prompt that would elicit offensive content", we add an affirmative prefix (e.g., "I have an idea for a prompt:") to the LLM's response. This encourages the LLM to continue generating the rest of the response, leading to sampling harmful instructions. Another LLM generates a response to the harmful instruction, and the teacher model labels the instruction-response pair. We empirically show that our HarmAug outperforms other relevant baselines. Moreover, a 435-million-parameter safety guard model trained with HarmAug achieves an F1 score comparable to larger models with over 7 billion parameters, and even outperforms them in AUPRC, while operating at less than 25% of their computational cost.

2025-04-22

International Conference on Learning Representations (Accept (Poster))

MAP: Low-Compute Model Merging with Amortized Pareto Fronts via Quadratic Approximation

Zhiqi Bu

Huan He

Yonghui Wu

Jiang Bian

Yong Chen

Model merging has emerged as an effective approach to combine multiple single-task models into a multitask model. This process typically inv… (voir plus)olves computing a weighted average of the model parameters without any additional training. Existing model-merging methods focus on enhancing average task accuracy. However, interference and conflicts between the objectives of different tasks can lead to trade-offs during the merging process. In real-world applications, a set of solutions with various trade-offs can be more informative, helping practitioners make decisions based on diverse preferences. In this paper, we introduce a novel and low-compute algorithm, Model Merging with Amortized Pareto Front (MAP). MAP efficiently identifies a Pareto set of scaling coefficients for merging multiple models, reflecting the trade-offs involved. It amortizes the substantial computational cost of evaluations needed to estimate the Pareto front by using quadratic approximation surrogate models derived from a pre-selected set of scaling coefficients. Experimental results on vision and natural language processing tasks demonstrate that MAP can accurately identify the Pareto front, providing practitioners with flexible solutions to balance competing task objectives. We also introduce Bayesian MAP for scenarios with a relatively low number of tasks and Nested MAP for situations with a high number of tasks, further reducing the computational cost of evaluation.

2025-04-22

International Conference on Learning Representations (Accept (Poster))

Steering Masked Discrete Diffusion Models via Discrete Denoising Posterior Prediction

Jarrid Rector-Brooks

Zhangzhi Peng

Zachary Quinn

Michael Bronstein

Pranam Chatterjee

Avishek Joey Bose

Generative modeling of discrete data underlies important applications spanning text-based agents like ChatGPT to the design of the very buil… (voir plus)ding blocks of life in protein sequences. However, application domains need to exert control over the generated data by steering the generative process - typically via RLHF - to satisfy a specified property, reward, or affinity metric. In this paper, we study the problem of steering Masked Diffusion Models (MDMs), a recent class of discrete diffusion models that offer a compelling alternative to traditional autoregressive models. We introduce Discrete Denoising Posterior Prediction (DDPP), a novel framework that casts the task of steering pre-trained MDMs as a problem of probabilistic inference by learning to sample from a target Bayesian posterior. Our DDPP framework leads to a family of three novel objectives that are all simulation-free, and thus scalable while applying to general non-differentiable reward functions. Empirically, we instantiate DDPP by steering MDMs to perform class-conditional pixel-level image modeling, RLHF-based alignment of MDMs using text-based rewards, and finetuning protein language models to generate more diverse secondary structures and shorter proteins. We substantiate our designs via wet-lab validation, where we observe transient expression of reward-optimized protein sequences.

2025-04-22

International Conference on Learning Representations (Accept (Poster))

Structure Language Models for Protein Conformation Generation

Stephen Z. Lu

Hongyu Guo

Proteins adopt multiple structural conformations to perform their diverse biological functions, and understanding these conformations is cru… (voir plus)cial for advancing drug discovery. Traditional physics-based simulation methods often struggle with sampling equilibrium conformations and are computationally expensive. Recently, deep generative models have shown promise in generating protein conformations as a more efficient alternative. However, these methods predominantly rely on the diffusion process within a 3D geometric space, which typically centers around the vicinity of metastable states and is often inefficient in terms of runtime. In this paper, we introduce Structure Language Modeling (SLM) as a novel framework for efficient protein conformation generation. Specifically, the protein structures are first encoded into a compact latent space using a discrete variational auto-encoder, followed by conditional language modeling that effectively captures sequence-specific conformation distributions. This enables a more efficient and interpretable exploration of diverse ensemble modes compared to existing methods. Based on this general framework, we instantiate SLM with various popular LM architectures as well as proposing the ESMDiff, a novel BERT-like structure language model fine-tuned from ESM3 with masked diffusion. We verify our approach in various scenarios, including the equilibrium dynamics of BPTI, conformational change pairs, and intrinsically disordered proteins. SLM provides a highly efficient solution, offering a 20-100x speedup than existing methods in generating diverse conformations, shedding light on promising avenues for future research.

2025-04-22

ICLR (Accept (Poster))

Towards Improving Exploration Through Sibling Augmented GFlowNets

Emmanuel Bengio

2025-04-22

International Conference on Learning Representations (Accept (Poster))

VCR: A Task for Pixel-Level Complex Reasoning in Vision Language Models via Restoring Occluded Text

Ge Zhang

Perouz Taslakian

Sai Rajeswar

We introduce Visual Caption Restoration (VCR), a novel vision-language task that challenges models to accurately restore partially obscured … (voir plus)texts using pixel-level hints within images. This task stems from the observation that text embedded in images is intrinsically different from common visual elements and natural language due to the need to align the modalities of vision, text, and text embedded in images. While numerous works have integrated text embedded in images into visual question-answering tasks, approaches to these tasks generally rely on optical character recognition or masked language modeling, thus reducing the task to mainly text-based processing. However, text-based processing becomes ineffective in VCR as accurate text restoration depends on the combined information from provided images, context, and subtle cues from the tiny exposed areas of masked texts. We develop a pipeline to generate synthetic images for the VCR task using image-caption pairs, with adjustable caption visibility to control the task difficulty. With this pipeline, we construct a dataset for VCR called VCR-Wiki using images with captions from Wikipedia, comprising 2.11M English and 346K Chinese entities in both easy and hard split variants. Our results reveal that current vision language models significantly lag behind human performance in the VCR task, and merely fine-tuning the models on our dataset does not lead to notable improvements. We release VCR-Wiki and the data construction code to facilitate future research.

2025-04-22

International Conference on Learning Representations (Accept (Poster))

Open Problems in Technical AI Governance

Anka Reuel

Benjamin Bucknall

Stephen Casper

Timothy Fist

Lisa Soder

Onni Aarne

Lewis Hammond

Lujain Ibrahim

Peter Wills

Markus Anderljung

Ben Garfinkel

Lennart Heim

Andrew Trask

Gabriel Mukobi

Rylan Schaeffer

Mauricio Baker

Sara Hooker

Irene Solaiman

Sasha Luccioni … (voir 14 de plus)

Alexandra Luccioni

Nitarshan Rajkumar

Nicolas Moës

Jeffrey Ladish

David Bau

Paul Bricman

Neel Guha

Jessica Newman

Tobin South

Alex Pentland

Sanmi Koyejo

Mykel Kochenderfer

Robert Trager

AI progress is creating a growing range of risks and opportunities, but it is often unclear how they should be navigated. In many cases, the… (voir plus) barriers and uncertainties faced are at least partly technical. Technical AI governance, referring to technical analysis and tools for supporting the effective governance of AI, seeks to address such challenges. It can help to (a) identify areas where intervention is needed, (b) identify and assess the efficacy of potential governance actions, and (c) enhance governance options by designing mechanisms for enforcement, incentivization, or compliance. In this paper, we explain what technical AI governance is, why it is important, and present a taxonomy and incomplete catalog of its open problems. This paper is intended as a resource for technical researchers or research funders looking to contribute to AI governance.

2025-04-13

Transactions on Machine Learning Research (accepté)