Yoshua Bengio

Mise à l’échelle au service du raisonnement et de l’apprentissage automatique basé sur un modèle

Scaling in the service of reasoning & model-based ML

4 avril 2023

par

Yoshua Bengio

Edward J. Hu

Une collaboration entre Mila et Relation Therapeutics pour découvrir in vitro de nouvelles associations médicamenteuses synergiques

A collaboration between Mila and Relation Therapeutics to discover novel synergistic combinations of drugs in vitro

23 mars 2022

par

Paul Bertin

Jake P. Taylor-King

Yoshua Bengio

Les réseaux de flot génératifs

15 mars 2022

par

Yoshua Bengio

Publications

Steering Masked Discrete Diffusion Models via Discrete Denoising Posterior Prediction

Jarrid Rector-Brooks

Mohsin Hasan

Zhangzhi Peng

Zachary Quinn

Michael Bronstein

Pranam Chatterjee

Avishek Joey Bose

Generative modeling of discrete data underlies important applications spanning text-based agents like ChatGPT to the design of the very buil… (voir plus)ding blocks of life in protein sequences. However, application domains need to exert control over the generated data by steering the generative process - typically via RLHF - to satisfy a specified property, reward, or affinity metric. In this paper, we study the problem of steering Masked Diffusion Models (MDMs), a recent class of discrete diffusion models that offer a compelling alternative to traditional autoregressive models. We introduce Discrete Denoising Posterior Prediction (DDPP), a novel framework that casts the task of steering pre-trained MDMs as a problem of probabilistic inference by learning to sample from a target Bayesian posterior. Our DDPP framework leads to a family of three novel objectives that are all simulation-free, and thus scalable while applying to general non-differentiable reward functions. Empirically, we instantiate DDPP by steering MDMs to perform class-conditional pixel-level image modeling, RLHF-based alignment of MDMs using text-based rewards, and finetuning protein language models to generate more diverse secondary structures and shorter proteins. We substantiate our designs via wet-lab validation, where we observe transient expression of reward-optimized protein sequences.

2025-04-22

International Conference on Learning Representations (Accept (Poster))

Open Problems in Technical AI Governance

Anka Reuel

Benjamin Bucknall

Stephen Casper

Timothy Fist

Lisa Soder

Onni Aarne

Lewis Hammond

Lujain Ibrahim

Alan Chan

Peter Wills

Markus Anderljung

Ben Garfinkel

Lennart Heim

Andrew Trask

Gabriel Mukobi

Rylan Schaeffer

Mauricio Baker

Sara Hooker

Irene Solaiman

Sasha Luccioni … (voir 14 de plus)

Alexandra Luccioni

Nitarshan Rajkumar

Nicolas Moës

Jeffrey Ladish

David Bau

Paul Bricman

Neel Guha

Jessica Newman

Tobin South

Alex Pentland

Sanmi Koyejo

Mykel Kochenderfer

Robert Trager

AI progress is creating a growing range of risks and opportunities, but it is often unclear how they should be navigated. In many cases, the… (voir plus) barriers and uncertainties faced are at least partly technical. Technical AI governance, referring to technical analysis and tools for supporting the effective governance of AI, seeks to address such challenges. It can help to (a) identify areas where intervention is needed, (b) identify and assess the efficacy of potential governance actions, and (c) enhance governance options by designing mechanisms for enforcement, incentivization, or compliance. In this paper, we explain what technical AI governance is, why it is important, and present a taxonomy and incomplete catalog of its open problems. This paper is intended as a resource for technical researchers or research funders looking to contribute to AI governance.

2025-04-13

Transactions on Machine Learning Research (accepté)

Assessing SAM for Tree Crown Instance Segmentation from Drone Imagery

Mélisande Teng

2025-03-25

ArXiv (prépublication)

Extendable Planning via Multiscale Diffusion

Chang Chen

Hany Hamed

Doojin Baek

Taegu Kang

Sungjin Ahn

Long-horizon planning is crucial in complex environments, but diffusion-based planners like Diffuser are limited by the trajectory lengths o… (voir plus)bserved during training. This creates a dilemma: long trajectories are needed for effective planning, yet they degrade model performance. In this paper, we introduce this extendable long-horizon planning challenge and propose a two-phase solution. First, Progressive Trajectory Extension incrementally constructs longer trajectories through multi-round compositional stitching. Second, the Hierarchical Multiscale Diffuser enables efficient training and inference over long horizons by reasoning across temporal scales. To avoid the need for multiple separate models, we propose Adaptive Plan Pondering and the Recursive HM-Diffuser, which unify hierarchical planning within a single model. Experiments show our approach yields strong performance gains, advancing scalable and efficient decision-making over long-horizons.

2025-03-24

ArXiv (prépublication)

A scalable gene network model of regulatory dynamics in single cells

Paul Bertin

Joseph D Viviano

Alejandro Tejada-Lapuerta

Weixu Wang

Stefan Bauer

Fabian J. Theis

2025-03-24

ArXiv (prépublication)

Offline Model-Based Optimization: Comprehensive Review

Jiayao Gu

Zixuan Liu

Can Chen

2025-03-20

ArXiv (prépublication)

What makes a theory of consciousness unscientific?

Derek H. Mark G. Tristan A. Yoshua James W. Jacob Dean D Arnold Baxter Bekinschtein Bengio Bisley Browning

Derek H. Arnold

Mark G. Baxter

Tristan A. Bekinschtein

Vincent Taschereau-Dumouchel

James W. Bisley

Jacob Browning

Dean Buonomano

David Carmel

Marisa Carrasco

Peter Carruthers

Olivia Carter

Dorita H. F. Chang

Ian Charest

Mouslim Cherkaoui

Axel Cleeremans

Michael A. Cohen

Philip R. Corlett

Kalina Christoff

Sam Cumming … (voir 84 de plus)

Cody A. Cushing

Beatrice de Gelder

Felipe De Brigard

Daniel C. Dennett

Nadine Dijkstra

Adrien Doerig

Paul E. Dux

Stephen M. Fleming

Keith Frankish

Chris D. Frith

Sarah Garfinkel

Melvyn A. Goodale

Jacqueline Gottlieb

Jake R. Hanson

Ran R. Hassin

Michael H. Herzog

Cecilia Heyes

Po-Jang Hsieh

Shao-Min Hung

Robert Kentridge

Tomas Knapen

Nikos Konstantinou

Konrad Kording

Timo L. Kvamme

Sze Chai Kwok

Renzo C. Lanfranco

Hakwan Lau

Joseph LeDoux

Alan L. F. Lee

Camilo Libedinsky

Matthew D. Lieberman

Ying-Tung Lin

Ka-Yuet Liu

Maro G. Machizawa

Julio Martinez-Trujillo

Janet Metcalfe

Matthias Michel

Kenneth D. Miller

Partha P. Mitra

Dean Mobbs

Robert M. Mok

Jorge Morales

Myrto Mylopoulos

Brian Odegaard

Charles C.-F. Or

Adrian M. Owen

David Pereplyotchik

Franco Pestilli

Megan A. K. Peters

Ian Phillips

Rosanne L. Rademaker

Dobromir Rahnev

Geraint Rees

Dario L. Ringach

Adina Roskies

Daniela Schiller

Aaron Schurger

D. Samuel Schwarzkopf

Ryan B. Scott

Aaron R. Seitz

Joshua Shepherd

Juha Silvanto

Heleen A. Slagter

Barry C. Smith

Guillermo Solovey

David Soto

Hugo Spiers

Timo Stein

Frank Tong

Peter U. Tse

Jonas Vibell

Sebastian Watzl

Taylor Webb

Josh Weisberg

Thalia Wheatley

Michael H. Herzog

Martijn E. Wokke

Hakwan Lau

Michał Klincewicz

Tony Cheng

Michael Schmitz

Miguel Ángel Sebastián

Joel S. Snyder

2025-03-09

Nature Neuroscience (inconnu)

Shaping Inductive Bias in Diffusion Models through Frequency-Based Noise Control

Thomas Jiralerspong

Berton Earnshaw

Jason Hartford

Luca Scimeca

Diffusion Probabilistic Models (DPMs) are powerful generative models that have achieved unparalleled success in a number of generative tasks… (voir plus). In this work, we aim to build inductive biases into the training and sampling of diffusion models to better accommodate the target distribution of the data to model. For topologically structured data, we devise a frequency-based noising operator to purposefully manipulate, and set, these inductive biases. We first show that appropriate manipulations of the noising forward process can lead DPMs to focus on particular aspects of the distribution to learn. We show that different datasets necessitate different inductive biases, and that appropriate frequency-based noise control induces increased generative performance compared to standard diffusion. Finally, we demonstrate the possibility of ignoring information at particular frequencies while learning. We show this in an image corruption and recovery task, where we train a DPM to recover the original target distribution after severe noise corruption.

2025-03-05

ICLR.cc/2025/Workshop/DeLTa (poster)

Laurence Perreault-Levasseur

Solving Bayesian Inverse Problems with Diffusion Priors and Off-Policy RL

Glen Berseth

Nikolay Malkin

This paper presents a practical application of Relative Trajectory Balance (RTB), a recently introduced off-policy reinforcement learning (R… (voir plus)L) objective that can asymptotically solve Bayesian inverse problems optimally. We extend the original work by using RTB to train conditional diffusion model posteriors from pretrained unconditional priors for challenging linear and non-linear inverse problems in vision, and science. We use the objective alongside techniques such as off-policy backtracking exploration to improve training. Importantly, our results show that existing training-free diffusion posterior methods struggle to perform effective posterior inference in latent space due to inherent biases.

2025-03-05

ICLR.cc/2025/Workshop/DeLTa (poster)

AlignVLM: Bridging Vision and Language Latent Spaces for Multimodal Document Understanding

Juan A. Rodriguez

Chao Wang

Akshay Kalkunte Suresh

Abhay Puri

Xiangru Jian

Pierre-Andre Noel

Sathwik Tejaswi Madhusudhan

Enamul Hoque

Christopher Pal

Issam H. Laradji

David Vázquez

Perouz Taslakian … (voir 2 de plus)

Spandana Gella

Sai Rajeswar

Aligning visual features with language embeddings is a key challenge in vision-language models (VLMs). The performance of such models hinges… (voir plus) on having a good connector that maps visual features generated by a vision encoder to a shared embedding space with the LLM while preserving semantic similarity. Existing connectors, such as multilayer perceptrons (MLPs), lack inductive bias to constrain visual features within the linguistic structure of the LLM's embedding space, making them data-hungry and prone to cross-modal misalignment. In this work, we propose a novel vision-text alignment method, AlignVLM, that maps visual features to a weighted average of LLM text embeddings. Our approach leverages the linguistic priors encoded by the LLM to ensure that visual features are mapped to regions of the space that the LLM can effectively interpret. AlignVLM is particularly effective for document understanding tasks, where visual and textual modalities are highly correlated. Our extensive experiments show that AlignVLM achieves state-of-the-art performance compared to prior alignment methods, with larger gains on document understanding tasks and under low-resource setups. We provide further analysis demonstrating its efficiency and robustness to noise.

2025-03-04

ICLR.cc/2025/Workshop/Re-Align (poster)

Learning Decision Trees as Amortized Structure Inference

Mohammed Mahfoud

Ghait Boukachab

Michał Koziarski

Alex Hernandez-Garcia

Stefan Bauer

Nikolay Malkin

2025-03-04

ICLR.cc/2025/Workshop/FPI (poster)

EarthView: A Large Scale Remote Sensing Dataset for Self-Supervision

Diego Velazquez

Pau Rodríguez

Sergio Alonso

Josep M. Gonfaus

Jordi Gonzalez

Gerardo Richarte

Javier Marin

Alexandre Lacoste

This paper presents EarthView, a comprehensive dataset specifically designed for self-supervision on remote sensing data, intended to enhanc… (voir plus)e deep learning applications on Earth monitoring tasks. The dataset spans 15 tera pixels of global remote-sensing data, combining imagery from a diverse range of sources, including NEON, Sentinel, and a novel release of 1m spatial resolution data from Satellogic. Our dataset provides a wide spectrum of image data with varying resolutions, harnessed from different sensors and organized coherently into an accessible HuggingFace dataset in parquet format. This data spans five years, from 2017 to 2022. Accompanying the dataset, we introduce EarthMAE, a tailored Masked Autoencoder, developed to tackle the distinct challenges of remote sensing data. Trained in a self-supervised fashion, EarthMAE effectively processes different data modalities such as hyperspectral, multispectral, topographical data, segmentation maps, and temporal structure. This model helps us show that pre-training on Satellogic data improves performance on downstream tasks. While there is still a gap to fill in MAE for heterogeneous data, we regard this innovative combination of an expansive, diverse dataset and a versatile model adapted for self-supervised learning as a stride forward in deep learning for Earth monitoring.

2025-03-03

2025 IEEE/CVF Winter Conference on Applications of Computer Vision Workshops (WACVW) (publié)