Yoshua Bengio

jamal.abouhaibeh@mila.quebec

Biographie

*Pour toute demande média, veuillez écrire à medias@mila.quebec.

Pour plus d’information, contactez Cassidy MacNeil, adjointe principale et responsable des opérations cassidy.macneil@mila.quebec.

Reconnu comme une sommité mondiale en intelligence artificielle, Yoshua Bengio s’est surtout distingué par son rôle de pionnier en apprentissage profond, ce qui lui a valu le prix A. M. Turing 2018, le « prix Nobel de l’informatique », avec Geoffrey Hinton et Yann LeCun. Il est professeur titulaire à l’Université de Montréal, fondateur et conseiller scientifique de Mila – Institut québécois d’intelligence artificielle, et codirige en tant que senior fellow le programme Apprentissage automatique, apprentissage biologique de l'Institut canadien de recherches avancées (CIFAR). Il occupe également la fonction de conseiller spécial et directeur scientifique fondateur d’IVADO.

En 2018, il a été l’informaticien qui a recueilli le plus grand nombre de nouvelles citations au monde. En 2019, il s’est vu décerner le prestigieux prix Killam. Depuis 2022, il détient le plus grand facteur d’impact (h-index) en informatique à l’échelle mondiale. Il est fellow de la Royal Society de Londres et de la Société royale du Canada, et officier de l’Ordre du Canada.

Soucieux des répercussions sociales de l’IA et de l’objectif que l’IA bénéficie à tous, il a contribué activement à la Déclaration de Montréal pour un développement responsable de l’intelligence artificielle.

Étudiants actuels

Jamal Abou Haibeh

Collaborateur·rice alumni - McGill

Collaborateur·rice de recherche - Cambridge University

berkes.anais@mila.quebec

sanghyeok.choi@mila.quebec

Rim Assouel

Doctorat - UdeM

assouelr@mila.quebec

Xiaoyin Chen

Doctorat - UdeM

xiaoyin.chen@mila.quebec

Sanghyeok Choi

Collaborateur·rice de recherche - KAIST

Doctorat - UdeM

aniket.didolkar@mila.quebec

Doctorat - UdeM

eric.elmoznino@mila.quebec

Doctorat - UdeM

lena-nehale.ezzine@mila.quebec

Jean-Pierre Falet

Doctorat - UdeM

jean-pierre.falet@mila.quebec

Doctorat

pietro.greiner@mila.quebec

Mohsin Hasan

Doctorat - UdeM

mohsin.hasan@mila.quebec

thomas.jiralerspong@mila.quebec

Moksh Jain

Doctorat - UdeM

moksh.jain@mila.quebec

Doctorat - UdeM

Collaborateur·rice alumni - UdeM

younesse.kaddar@mila.quebec

Hyeonah Kim

Postdoctorat - UdeM

hyeonah.kim@mila.quebec

Minsu Kim

Collaborateur·rice de recherche - UdeM

minsu.kim@mila.quebec

Collaborateur·rice de recherche - KOREA ADVANCED INSTITUTE OF SCIENCE AND TECHNOLOGY

jeongjae.lee@mila.quebec

cristian-dragos.manta@mila.quebec

Song LIU

Collaborateur·rice de recherche - s.o.

Collaborateur·rice de recherche

Cristian Dragos Manta

Doctorat - UdeM

Doctorat - UdeM

Visiteur de recherche indépendant - UdeM

nguyenph@mila.quebec

Ali Parviz

Collaborateur·rice de recherche - Ying Wu Coll of Computing

ali.parviz@mila.quebec

Lena Podina

Collaborateur·rice de recherche - University of Waterloo

elena.podina@mila.quebec

jarrid.rector-brooks@mila.quebec

Jarrid Rector-Brooks

Doctorat - UdeM

Danyal REHMAN

Postdoctorat - UdeM

danyal.rehman@mila.quebec

oliver.richardson@mila.quebec

Oli RICHARDSON

Postdoctorat - UdeM

Camille Rochefort-Boulanger

Doctorat - UdeM

rochefoc@mila.quebec

Divya Sharma

Postdoctorat - UdeM

divya.sharma@mila.quebec

Mélisande Astrid Crystal Teng

Collaborateur·rice alumni - UdeM

tengmeli@mila.quebec

alexander.tong@mila.quebec

Alex Tong

Collaborateur·rice alumni - UdeM

Collaborateur·rice de recherche - University of Cambridge

oszkar.urban@mila.quebec

donna.vakalis@mila.quebec

Donna Vakalis

Collaborateur·rice alumni - UdeM

Doctorat - UdeM

siddarth.venkatraman@mila.quebec

Collaborateur·rice de recherche

joseph.viviano2@mila.quebec

Collaborateur·rice de recherche - UdeM

esmeralda.whitammer@mila.quebec

Collaborateur·rice alumni

omar.younis@mila.quebec

Collaborateur·rice de recherche - UdeM

taeyoung.yun@mila.quebec

Skipper : combiner l’abstraction spatiale et temporelle afin d’améliorer la généralisation

Tianyu Zhang

Doctorat - UdeM

tianyu.zhang@mila.quebec

Doctorat - McGill

Harry Zhao

Collaborateur·rice alumni - McGill

Billets de blogue

Generic thumbnail for Mila Blog articles.

22 février 2024

par

Mingde Harry Zhao

Safa Alver

Harm van Seijen

Romain Laroche

Doina Precup

Yoshua Bengio

Mise à l’échelle au service du raisonnement et de l’apprentissage automatique basé sur un modèle

Scaling in the service of reasoning & model-based ML

4 avril 2023

par

Yoshua Bengio

Edward J. Hu

Une collaboration entre Mila et Relation Therapeutics pour découvrir in vitro de nouvelles associations médicamenteuses synergiques

A collaboration between Mila and Relation Therapeutics to discover novel synergistic combinations of drugs in vitro

23 mars 2022

par

Paul Bertin

Jake P. Taylor-King

Yoshua Bengio

Les réseaux de flot génératifs

15 mars 2022

par

Yoshua Bengio

Publications

Benchmarking Bayesian Causal Discovery Methods for Downstream Treatment Effect Estimation

2023-06-18

ICML.cc/2023/Workshop/SPIGM (poster)

Thompson Sampling for Improved Exploration in GFlowNets

Jarrid Rector-Brooks

Kanika Madan

Moksh J. Jain

Maksym Korablyov

Cheng-Hao Liu

A. Chandar

Nikolay Malkin

Generative flow networks (GFlowNets) are amortized variational inference algorithms that treat sampling from a distribution over composition… (voir plus)al objects as a sequential decision-making problem with a learnable action policy. Unlike other algorithms for hierarchical sampling that optimize a variational bound, GFlowNet algorithms can stably run off-policy, which can be advantageous for discovering modes of the target distribution. Despite this flexibility in the choice of behaviour policy, the optimal way of efficiently selecting trajectories for training has not yet been systematically explored. In this paper, we view the choice of trajectories for training as an active learning problem and approach it using Bayesian techniques inspired by methods for multi-armed bandits. The proposed algorithm, Thompson sampling GFlowNets (TS-GFN), maintains an approximate posterior distribution over policies and samples trajectories from this posterior for training. We show in two domains that TS-GFN yields improved exploration and thus faster convergence to the target distribution than the off-policy exploration strategies used in past work.

2023-06-18

ICML.cc/2023/Workshop/SPIGM (poster)

Spotlight Attention: Robust Object-Centric Learning With a Spatial Locality Prior

Ayush Chakravarthy

Trang Nguyen

Anirudh Goyal

Michael C. Mozer

2023-05-30

arXiv (prépublication)

arxiv.org

Model evaluation for extreme risks

Toby Shevlane

Sebastian Farquhar

Ben Garfinkel

Mary Phuong

Jess Whittlestone

Jade Leung

Daniel Kokotajlo

Nahema A. Marchal

Markus Anderljung

Noam Kolt

Lewis Ho

Divya Siddarth

Shahar Avin

W. Hawkins

Been Kim

Iason Gabriel

Vijay Bolina

Jack Clark

Paul F. Christiano … (voir 1 de plus)

Allan Dafoe

2023-05-23

ArXiv (prépublication)

arxiv.org

Responses of pyramidal cell somata and apical dendrites in mouse visual cortex over multiple days

Colleen J. Gillon

Jérôme A. Lecoq

Jason E. Pina

Ruweida Ahmed

Yazan N. Billeh

Shiella Caldejon

Peter Groblewski

Timothy M. Henley

India Kato

Eric Lee

Jennifer Luviano

Kyla Mace

Chelsea Nayan

Thuyanh V. Nguyen

Kat North

Jed Perkins

Sam Seid

Matthew T. Valley

Ali Williford

Yoshua Bengio … (voir 3 de plus)

Timothy P. Lillicrap

Joel Zylberberg

Blake A. Richards

The apical dendrites of pyramidal neurons in sensory cortex receive primarily top-down signals from associative and motor regions, while cel… (voir plus)l bodies and nearby dendrites are heavily targeted by locally recurrent or bottom-up inputs from the sensory periphery. Based on these differences, a number of theories in computational neuroscience postulate a unique role for apical dendrites in learning. However, due to technical challenges in data collection, little data is available for comparing the responses of apical dendrites to cell bodies over multiple days. Here we present a dataset collected through the Allen Institute Mindscope’s OpenScope program that addresses this need. This dataset comprises high-quality two-photon calcium imaging from the apical dendrites and the cell bodies of visual cortical pyramidal neurons, acquired over multiple days in awake, behaving mice that were presented with visual stimuli. Many of the cell bodies and dendrite segments were tracked over days, enabling analyses of how their responses change over time. This dataset allows neuroscientists to explore the differences between apical and somatic processing and plasticity.

2023-05-16

Scientific Data (publié)

Automated Detection of Anatomical Landmarks During Colonoscopy Using a Deep Learning Model

Mahsa Taghiakbari

Sina Hamidi Ghalehjegh

Emmanuel Jehanno

Tess Berthier

Lisa Di Jorio

Saber Ghadakzadeh

Alan Barkun

Mark Takla

Mickael Bouin

Eric Deslandres

Simon Bouchard

Sacha Sidani

Daniel von Renteln

Abstract Background and aims Identification and photo-documentation of the ileocecal valve (ICV) and appendiceal orifice (AO) confirm comple… (voir plus)teness of colonoscopy examinations. We aimed to develop and test a deep convolutional neural network (DCNN) model that can automatically identify ICV and AO, and differentiate these landmarks from normal mucosa and colorectal polyps. Methods We prospectively collected annotated full-length colonoscopy videos of 318 patients undergoing outpatient colonoscopies. We created three nonoverlapping training, validation, and test data sets with 25,444 unaltered frames extracted from the colonoscopy videos showing four landmarks/image classes (AO, ICV, normal mucosa, and polyps). A DCNN classification model was developed, validated, and tested in separate data sets of images containing the four different landmarks. Results After training and validation, the DCNN model could identify both AO and ICV in 18 out of 21 patients (85.7%). The accuracy of the model for differentiating AO from normal mucosa, and ICV from normal mucosa were 86.4% (95% CI 84.1% to 88.5%), and 86.4% (95% CI 84.1% to 88.6%), respectively. Furthermore, the accuracy of the model for differentiating polyps from normal mucosa was 88.6% (95% CI 86.6% to 90.3%). Conclusion This model offers a novel tool to assist endoscopists with automated identification of AO and ICV during colonoscopy. The model can reliably distinguish these anatomical landmarks from normal mucosa and colorectal polyps. It can be implemented into automated colonoscopy report generation, photo-documentation, and quality auditing solutions to improve colonoscopy reporting quality.

2023-05-01

Journal of the Canadian Association of Gastroenterology (publié)

GFlowNets and Variational Inference

Nikolay Malkin

This paper builds bridges between two families of probabilistic algorithms: (hierarchical) variational inference (VI), which is typically us… (voir plus)ed to model distributions over continuous spaces, and generative flow networks (GFlowNets), which have been used for distributions over discrete structures such as graphs. We demonstrate that, in certain cases, VI algorithms are equivalent to special cases of GFlowNets in the sense of equality of expected gradients of their learning objectives. We then point out the differences between the two families and show how these differences emerge experimentally. Notably, GFlowNets, which borrow ideas from reinforcement learning, are more amenable than VI to off-policy training without the cost of high gradient variance induced by importance sampling. We argue that this property of GFlowNets can provide advantages for capturing diversity in multimodal target distributions.

2023-04-30

International Conference on Learning Representations (Accept: poster)

Latent Bottlenecked Attentive Neural Processes

Leo Feng

Hossein Hajimirsadeghi

Mohamed Osama Ahmed

Neural Processes (NPs) are popular methods in meta-learning that can estimate predictive uncertainty on target datapoints by conditioning on… (voir plus) a context dataset. Previous state-of-the-art method Transformer Neural Processes (TNPs) achieve strong performance but require quadratic computation with respect to the number of context datapoints, significantly limiting its scalability. Conversely, existing sub-quadratic NP variants perform significantly worse than that of TNPs. Tackling this issue, we propose Latent Bottlenecked Attentive Neural Processes (LBANPs), a new computationally efficient sub-quadratic NP variant, that has a querying computational complexity independent of the number of context datapoints. The model encodes the context dataset into a constant number of latent vectors on which self-attention is performed. When making predictions, the model retrieves higher-order information from the context dataset via multiple cross-attention mechanisms on the latent vectors. We empirically show that LBANPs achieve results competitive with the state-of-the-art on meta-regression, image completion, and contextual multi-armed bandits. We demonstrate that LBANPs can trade-off the computational cost and performance according to the number of latent vectors. Finally, we show LBANPs can scale beyond existing attention-based NP variants to larger dataset settings.

2023-04-30

International Conference on Learning Representations (Accept: poster)

Latent State Marginalization as a Low-Cost Approach for Improving Exploration

Qinqing Zheng

Ricky T. Q. Chen

While the maximum entropy (MaxEnt) reinforcement learning (RL) framework -- often touted for its exploration and robustness capabilities -- … (voir plus)is usually motivated from a probabilistic perspective, the use of deep probabilistic models has not gained much traction in practice due to their inherent complexity. In this work, we propose the adoption of latent variable policies within the MaxEnt framework, which we show can provably approximate any policy distribution, and additionally, naturally emerges under the use of world models with a latent belief state. We discuss why latent variable policies are difficult to train, how naive approaches can fail, then subsequently introduce a series of improvements centered around low-cost marginalization of the latent state, allowing us to make full use of the latent state at minimal additional cost. We instantiate our method under the actor-critic framework, marginalizing both the actor and critic. The resulting algorithm, referred to as Stochastic Marginal Actor-Critic (SMAC), is simple yet effective. We experimentally validate our method on continuous control tasks, showing that effective marginalization can lead to better exploration and more robust training. Our implementation is open sourced at https://github.com/zdhNarsil/Stochastic-Marginal-Actor-Critic.

2023-04-30

International Conference on Learning Representations (Accept: poster)

Predictive Inference with Feature Conformal Prediction

Jiaye Teng

Chuan Wen

Dinghuai Zhang

Yang Gao

Yang Yuan

Conformal prediction is a distribution-free technique for establishing valid prediction intervals. Although conventionally people conduct co… (voir plus)nformal prediction in the output space, this is not the only possibility. In this paper, we propose feature conformal prediction, which extends the scope of conformal prediction to semantic feature spaces by leveraging the inductive bias of deep representation learning. From a theoretical perspective, we demonstrate that feature conformal prediction provably outperforms regular conformal prediction under mild assumptions. Our approach could be combined with not only vanilla conformal prediction, but also other adaptive conformal prediction methods. Apart from experiments on existing predictive inference benchmarks, we also demonstrate the state-of-the-art performance of the proposed methods on large-scale tasks such as ImageNet classification and Cityscapes image segmentation.The code is available at https://github.com/AlvinWen428/FeatureCP.

2023-04-30

International Conference on Learning Representations (Accept: poster)

Robust and Controllable Object-Centric Learning through Energy-based Models

Ruixiang Zhang

Tong Che

Boris Ivanovic

Renhao Wang

Marco Pavone

Liam Paull

Humans are remarkably good at understanding and reasoning about complex visual scenes. The capability to decompose low-level observations in… (voir plus)to discrete objects allows us to build a grounded abstract representation and identify the compositional structure of the world. Accordingly, it is a crucial step for machine learning models to be capable of inferring objects and their properties from visual scenes without explicit supervision. However, existing works on object-centric representation learning either rely on tailor-made neural network modules or strong probabilistic assumptions in the underlying generative and inference processes. In this work, we present \ours, a conceptually simple and general approach to learning object-centric representations through an energy-based model. By forming a permutation-invariant energy function using vanilla attention blocks readily available in Transformers, we can infer object-centric latent variables via gradient-based MCMC methods where permutation equivariance is automatically guaranteed. We show that \ours can be easily integrated into existing architectures and can effectively extract high-quality object-centric representations, leading to better segmentation accuracy and competitive downstream task performance. Further, empirical evaluations show that \ours's learned representations are robust against distribution shift. Finally, we demonstrate the effectiveness of \ours in systematic compositional generalization, by re-composing learned energy functions for novel scene generation and manipulation.

2023-04-30

International Conference on Learning Representations (Accept: poster)

Stateful Active Facilitator: Coordination and Environmental Heterogeneity in Cooperative Multi-Agent Reinforcement Learning

Dianbo Liu

Tianmin Shu

Michael Mozer

Nicolas Heess

In cooperative multi-agent reinforcement learning, a team of agents works together to achieve a common goal. Different environments or tasks… (voir plus) may require varying degrees of coordination among agents in order to achieve the goal in an optimal way. The nature of coordination will depend on the properties of the environment -- its spatial layout, distribution of obstacles, dynamics, etc. We term this variation of properties within an environment as heterogeneity. Existing literature has not sufficiently addressed the fact that different environments may have different levels of heterogeneity. We formalize the notions of coordination level and heterogeneity level of an environment and present HECOGrid, a suite of multi-agent RL environments that facilitates empirical evaluation of different MARL approaches across different levels of coordination and environmental heterogeneity by providing a quantitative control over coordination and heterogeneity levels of the environment. Further, we propose a Centralized Training Decentralized Execution learning approach called Stateful Active Facilitator (SAF) that enables agents to work efficiently in high-coordination and high-heterogeneity environments through a differentiable and shared knowledge source used during training and dynamic selection from a shared pool of policies. We evaluate SAF and compare its performance against baselines IPPO and MAPPO on HECOGrid. Our results show that SAF consistently outperforms the baselines across different tasks and different heterogeneity and coordination levels. We release the code for HECOGrid as well as all our experiments.

2023-04-30

ICLR (Accept: poster)