Yoshua Bengio

jamal.abouhaibeh@mila.quebec

Biographie

*Pour toute demande média, veuillez écrire à medias@mila.quebec.

Pour plus d’information, contactez Cassidy MacNeil, adjointe principale et responsable des opérations cassidy.macneil@mila.quebec.

Reconnu comme une sommité mondiale en intelligence artificielle, Yoshua Bengio s’est surtout distingué par son rôle de pionnier en apprentissage profond, ce qui lui a valu le prix A. M. Turing 2018, le « prix Nobel de l’informatique », avec Geoffrey Hinton et Yann LeCun. Il est professeur titulaire à l’Université de Montréal, fondateur et conseiller scientifique de Mila – Institut québécois d’intelligence artificielle, et codirige en tant que senior fellow le programme Apprentissage automatique, apprentissage biologique de l'Institut canadien de recherches avancées (CIFAR). Il occupe également la fonction de conseiller spécial et directeur scientifique fondateur d’IVADO.

En 2018, il a été l’informaticien qui a recueilli le plus grand nombre de nouvelles citations au monde. En 2019, il s’est vu décerner le prestigieux prix Killam. Depuis 2022, il détient le plus grand facteur d’impact (h-index) en informatique à l’échelle mondiale. Il est fellow de la Royal Society de Londres et de la Société royale du Canada, et officier de l’Ordre du Canada.

Soucieux des répercussions sociales de l’IA et de l’objectif que l’IA bénéficie à tous, il a contribué activement à la Déclaration de Montréal pour un développement responsable de l’intelligence artificielle.

Étudiants actuels

Jamal Abou Haibeh

Collaborateur·rice alumni - McGill

Berkes Anaïs

Collaborateur·rice de recherche - Cambridge University

Superviseur⋅e principal⋅e :

berkes.anais@mila.quebec

Rim Assouel

Doctorat - UdeM

assouelr@mila.quebec

Shahana Chatterjee

Collaborateur·rice de recherche - N/A

Superviseur⋅e principal⋅e :

shahana.chatterjee@mila.quebec

sanghyeok.choi@mila.quebec

Xiaoyin Chen

Doctorat - UdeM

xiaoyin.chen@mila.quebec

Sanghyeok Choi

Collaborateur·rice de recherche - KAIST

Doctorat - UdeM

aniket.didolkar@mila.quebec

Doctorat - UdeM

Co-superviseur⋅e :

Guillaume Lajoie

eric.elmoznino@mila.quebec

Doctorat - UdeM

lena-nehale.ezzine@mila.quebec

Jean-Pierre Falet

Doctorat - UdeM

jean-pierre.falet@mila.quebec

Doctorat

pietro.greiner@mila.quebec

Mohsin Hasan

Doctorat - UdeM

mohsin.hasan@mila.quebec

thomas.jiralerspong@mila.quebec

Moksh Jain

Doctorat - UdeM

moksh.jain@mila.quebec

Doctorat - UdeM

Superviseur⋅e principal⋅e :

Guillaume Lajoie

Collaborateur·rice alumni - UdeM

younesse.kaddar@mila.quebec

Minsu Kim

Collaborateur·rice de recherche - UdeM

minsu.kim@mila.quebec

Hyeonah Kim

Postdoctorat - UdeM

Superviseur⋅e principal⋅e :

Alex Hernández-García

hyeonah.kim@mila.quebec

Jeongjae Lee

Stagiaire de recherche - KOREA ADVANCED INSTITUTE OF SCIENCE AND TECHNOLOGY

jeongjae.lee@mila.quebec

Collaborateur·rice de recherche

Song LIU

Collaborateur·rice de recherche - s.o.

liusong299@gmail.com

cristian-dragos.manta@mila.quebec

Cristian Dragos Manta

Doctorat - UdeM

Co-superviseur⋅e :

Dhanya Sridhar

Sarthak Mittal

Doctorat - UdeM

Superviseur⋅e principal⋅e :

Visiteur de recherche indépendant - UdeM

nguyenph@mila.quebec

Ali Parviz

Collaborateur·rice de recherche - Ying Wu Coll of Computing

ali.parviz@mila.quebec

Lena Podina

Collaborateur·rice de recherche - University of Waterloo

Superviseur⋅e principal⋅e :

elena.podina@mila.quebec

jarrid.rector-brooks@mila.quebec

Jarrid Rector-Brooks

Doctorat - UdeM

Danyal REHMAN

Postdoctorat - UdeM

danyal.rehman@mila.quebec

oliver.richardson@mila.quebec

Oli RICHARDSON

Postdoctorat - UdeM

Camille Rochefort-Boulanger

Doctorat - UdeM

Superviseur⋅e principal⋅e :

Postdoctorat - UdeM

Co-superviseur⋅e :

Alex Hernández-García

divya.sharma@mila.quebec

Mélisande Astrid Crystal Teng

Collaborateur·rice alumni - UdeM

Co-superviseur⋅e :

Alex Tong

Collaborateur·rice alumni - UdeM

alexander.tong@mila.quebec

Collaborateur·rice de recherche - University of Cambridge

oszkar.urban@mila.quebec

Donna Vakalis

Collaborateur·rice alumni - UdeM

Co-superviseur⋅e :

donna.vakalis@mila.quebec

Doctorat - UdeM

Superviseur⋅e principal⋅e :

Glen Berseth

siddarth.venkatraman@mila.quebec

Collaborateur·rice de recherche - UdeM

esmeralda.whitammer@mila.quebec

Collaborateur·rice alumni

omar.younis@mila.quebec

Collaborateur·rice de recherche - UdeM

taeyoung.yun@mila.quebec

Skipper : combiner l’abstraction spatiale et temporelle afin d’améliorer la généralisation

Tianyu Zhang

Doctorat - UdeM

tianyu.zhang@mila.quebec

Doctorat - McGill

Superviseur⋅e principal⋅e :

Billets de blogue

Generic thumbnail for Mila Blog articles.

22 février 2024

par

Mingde Harry Zhao

Safa Alver

Harm van Seijen

Romain Laroche

Doina Precup

Yoshua Bengio

Mise à l’échelle au service du raisonnement et de l’apprentissage automatique basé sur un modèle

Scaling in the service of reasoning & model-based ML

4 avril 2023

par

Yoshua Bengio

Edward J. Hu

Une collaboration entre Mila et Relation Therapeutics pour découvrir in vitro de nouvelles associations médicamenteuses synergiques

A collaboration between Mila and Relation Therapeutics to discover novel synergistic combinations of drugs in vitro

23 mars 2022

par

Paul Bertin

Jake P. Taylor-King

Yoshua Bengio

Les réseaux de flot génératifs

15 mars 2022

par

Yoshua Bengio

Publications

PhAST: Physics-Aware, Scalable, and Task-Specific GNNs for Accelerated Catalyst Design

Alexandre Duval

Victor Schmidt

Santiago Miret

Alex Hernández-García

Mitigating the climate crisis requires a rapid transition towards lower-carbon energy. Catalyst materials play a crucial role in the electro… (voir plus)chemical reactions involved in numerous industrial processes key to this transition, such as renewable energy storage and electrofuel synthesis. To reduce the energy spent on such activities, we must quickly discover more efficient catalysts to drive electrochemical reactions. Machine learning (ML) holds the potential to efficiently model materials properties from large amounts of data, accelerating electrocatalyst design. The Open Catalyst Project OC20 dataset was constructed to that end. However, ML models trained on OC20 are still neither scalable nor accurate enough for practical applications. In this paper, we propose task-specific innovations applicable to most architectures, enhancing both computational efficiency and accuracy. This includes improvements in (1) the graph creation step, (2) atom representations, (3) the energy prediction head, and (4) the force prediction head. We describe these contributions, referred to as PhAST, and evaluate them thoroughly on multiple architectures. Overall, PhAST improves energy MAE by 4 to 42

2024-07-22

International Conference on Machine Learning (Accept (Poster))

Redesigning Information Markets in the Era of Language Models

Martin Weiss

Nasim Rahaman

Manuel Wuthrich

Li Erran Li

Bernhard Schölkopf

Christopher Pal

2024-07-09

colmweb.org/COLM/2024/Conference (accepté)

Memory Efficient Neural Processes via Constant Memory Attention Block

Leo Feng

Frederick Tung

Hossein Hajimirsadeghi

Mohamed Osama Ahmed

2024-07-07

Proceedings of the 41st International Conference on Machine Learning (publié)

proceedings.mlr.press

On Generalization for Generative Flow Networks

Anas Krichel

Nikolay Malkin

Salem Lahlou

Generative Flow Networks (GFlowNets) have emerged as an innovative learning paradigm designed to address the challenge of sampling from an u… (voir plus)nnormalized probability distribution, called the reward function. This framework learns a policy on a constructed graph, which enables sampling from an approximation of the target probability distribution through successive steps of sampling from the learned policy. To achieve this, GFlowNets can be trained with various objectives, each of which can lead to the model s ultimate goal. The aspirational strength of GFlowNets lies in their potential to discern intricate patterns within the reward function and their capacity to generalize effectively to novel, unseen parts of the reward function. This paper attempts to formalize generalization in the context of GFlowNets, to link generalization with stability, and also to design experiments that assess the capacity of these models to uncover unseen parts of the reward function. The experiments will focus on length generalization meaning generalization to states that can be constructed only by longer trajectories than those seen in training.

2024-07-02

ArXiv (prépublication)

Cell Morphology-Guided Small Molecule Generation with GFlowNets

Stephen Zhewen Lu

Ziqing Lu

Ehsan Hajiramezanali

Tommaso Biancalani

Gabriele Scalia

Michał Koziarski

2024-06-16

AI4Science @ International Conference on Machine Learning (poster)

Estimating Expectations without Sampling: Neural Stein Estimation

Mohsin Hasan

Dinghuai Zhang

Cheikh Ahmed

Awa Khouna

We propose a method for estimating the expected value of a given function …

2024-05-26

approximateinference.org/AABI/2024/Symposium (accepté)

Attention as an RNN

Leo Feng

Frederick Tung

Hossein Hajimirsadeghi

Mohamed Osama Ahmed

Greg Mori

The advent of Transformers marked a significant breakthrough in sequence modelling, providing a highly performant architecture capable of le… (voir plus)veraging GPU parallelism. However, Transformers are computationally expensive at inference time, limiting their applications, particularly in low-resource settings (e.g., mobile and embedded devices). Addressing this, we (1) begin by showing that attention can be viewed as a special Recurrent Neural Network (RNN) with the ability to compute its \textit{many-to-one} RNN output efficiently. We then (2) show that popular attention-based models such as Transformers can be viewed as RNN variants. However, unlike traditional RNNs (e.g., LSTMs), these models cannot be updated efficiently with new tokens, an important property in sequence modelling. Tackling this, we (3) introduce a new efficient method of computing attention's \textit{many-to-many} RNN output based on the parallel prefix scan algorithm. Building on the new attention formulation, we (4) introduce \textbf{Aaren}, an attention-based module that can not only (i) be trained in parallel (like Transformers) but also (ii) be updated efficiently with new tokens, requiring only constant memory for inferences (like traditional RNNs). Empirically, we show Aarens achieve comparable performance to Transformers on

2024-05-21

ArXiv (prépublication)

BitPruning: Learning Bitlengths for Aggressive and Accurate Quantization

Milos Nikolic

Ghouthi Boukli hacene

Ciaran Bannon

Alberto Delmas Lascorz

Matthieu Courbariaux

Omar Mohamed Awad

Isak Edo Vivancos

Vincent Gripon

Andreas Moshovos

Neural networks have demonstrably achieved state-of-the art accuracy using low-bitlength integer quantization, yielding both execution time … (voir plus)and energy benefits on existing hardware designs that support short bitlengths. However, the question of finding the minimum bitlength for a desired accuracy remains open. We introduce a training method for minimizing inference bitlength at any granularity while maintaining accuracy. Namely, we propose a regularizer that penalizes large bitlength representations throughout the architecture and show how it can be modified to minimize other quantifiable criteria, such as number of operations or memory footprint. We demonstrate that our method learns thrifty representations while maintaining accuracy. With ImageNet, the method produces an average per layer bitlength of 4.13, 3.76 and 4.36 bits on AlexNet, ResNet18 and MobileNet V2 respectively, remaining within 2.0%, 0.5% and 0.5% of the base TOP-1 accuracy.

2024-05-18

2024 IEEE International Symposium on Circuits and Systems (ISCAS) (publié)

Machine Learning and Information Theory Concepts Towards an AI Mathematician

Nikolay Malkin

The current state-of-the-art in artificial intelligence is impressive, especially in terms of mastery of language, but not so much in terms … (voir plus)of mathematical reasoning. What could be missing? Can we learn something useful about that gap from how the brains of mathematicians go about their craft? This essay builds on the idea that current deep learning mostly succeeds at system 1 abilities -- which correspond to our intuition and habitual behaviors -- but still lacks something important regarding system 2 abilities -- which include reasoning and robust uncertainty estimation. It takes an information-theoretical posture to ask questions about what constitutes an interesting mathematical statement, which could guide future work in crafting an AI mathematician. The focus is not on proving a given theorem but on discovering new and interesting conjectures. The central hypothesis is that a desirable body of theorems better summarizes the set of all provable statements, for example by having a small description length while at the same time being close (in terms of number of derivation steps) to many provable statements.

2024-05-14

Bulletin of the American Mathematical Society (publié)

Towards Guaranteed Safe AI: A Framework for Ensuring Robust and Reliable AI Systems

David Dalrymple

David

Joar Max Viktor Skalse

Stuart Russell

Max Tegmark

Sanjit A. Seshia

Steve Omohundro

Christian Szegedy

Ben Goldhaber

Nora Ammann

Alessandro Abate

Joe Halpern

Clark Barrett

Ding Zhao

Zhi-Xuan Tan

Jeannette Wing

Joshua B. Tenenbaum

Ensuring that AI systems reliably and robustly avoid harmful or dangerous behaviours is a crucial challenge, especially for AI systems with … (voir plus)a high degree of autonomy and general intelligence, or systems used in safety-critical contexts. In this paper, we will introduce and define a family of approaches to AI safety, which we will refer to as guaranteed safe (GS) AI. The core feature of these approaches is that they aim to produce AI systems which are equipped with high-assurance quantitative safety guarantees. This is achieved by the interplay of three core components: a world model (which provides a mathematical description of how the AI system affects the outside world), a safety specification (which is a mathematical description of what effects are acceptable), and a verifier (which provides an auditable proof certificate that the AI satisfies the safety specification relative to the world model). We outline a number of approaches for creating each of these three core components, describe the main technical challenges, and suggest a number of potential solutions to them. We also argue for the necessity of this approach to AI safety, and for the inadequacy of the main alternative approaches.

2024-05-09

ArXiv (prépublication)

Amortizing Intractable Inference in Large Language Models

Edward J. Hu

Nikolay Malkin

Autoregressive large language models (LLMs) compress knowledge from their training data through next-token conditional distributions. This l… (voir plus)imits tractable querying of this knowledge to start-to-end autoregressive sampling. However, many tasks of interest -- including sequence continuation, infilling, and other forms of constrained generation -- involve sampling from intractable posterior distributions. We address this limitation by using amortized Bayesian inference to sample from these intractable posteriors. Such amortization is algorithmically achieved by fine-tuning LLMs via diversity-seeking reinforcement learning algorithms: generative flow networks (GFlowNets). We empirically demonstrate that this distribution-matching paradigm of LLM fine-tuning can serve as an effective alternative to maximum-likelihood training and reward-maximizing policy optimization. As an important application, we interpret chain-of-thought reasoning as a latent variable modeling problem and demonstrate that our approach enables data-efficient adaptation of LLMs to tasks that require multi-step rationalization and tool use.

2024-05-06

International Conference on Learning Representations (Accept (oral))

Cycle Consistency Driven Object Discovery

Aniket Didolkar

Anirudh Goyal

Developing deep learning models that effectively learn object-centric representations, akin to human cognition, remains a challenging task. … (voir plus)Existing approaches facilitate object discovery by representing objects as fixed-size vectors, called ``slots'' or ``object files''. While these approaches have shown promise in certain scenarios, they still exhibit certain limitations. First, they rely on architectural priors which can be unreliable and usually require meticulous engineering to identify the correct objects. Second, there has been a notable gap in investigating the practical utility of these representations in downstream tasks. To address the first limitation, we introduce a method that explicitly optimizes the constraint that each object in a scene should be associated with a distinct slot. We formalize this constraint by introducing consistency objectives which are cyclic in nature. By integrating these consistency objectives into various existing slot-based object-centric methods, we showcase substantial improvements in object-discovery performance. These enhancements consistently hold true across both synthetic and real-world scenes, underscoring the effectiveness and adaptability of the proposed approach. To tackle the second limitation, we apply the learned object-centric representations from the proposed method to two downstream reinforcement learning tasks, demonstrating considerable performance enhancements compared to conventional slot-based and monolithic representation learning methods. Our results suggest that the proposed approach not only improves object discovery, but also provides richer features for downstream tasks.

2024-05-06

International Conference on Learning Representations (Accept (poster))