Yoshua Bengio

jamal.abouhaibeh@mila.quebec

Biography

*For media requests, please write to medias@mila.quebec.

For more information please contact Cassidy MacNeil, Senior Assistant and Operation Lead at cassidy.macneil@mila.quebec.

Yoshua Bengio is recognized worldwide as a leading expert in AI. He is most known for his pioneering work in deep learning, which earned him the 2018 A.M. Turing Award, “the Nobel Prize of computing,” with Geoffrey Hinton and Yann LeCun.

Bengio is a full professor at Université de Montréal, and the founder and scientific advisor of Mila – Quebec Artificial Intelligence Institute. He is also a senior fellow at CIFAR and co-directs its Learning in Machines & Brains program, serves as special advisor and founding scientific director of IVADO, and holds a Canada CIFAR AI Chair.

In 2019, Bengio was awarded the prestigious Killam Prize and in 2022, he was the most cited computer scientist in the world by h-index. He is a Fellow of the Royal Society of London, Fellow of the Royal Society of Canada, Knight of the Legion of Honor of France and Officer of the Order of Canada. In 2023, he was appointed to the UN’s Scientific Advisory Board for Independent Advice on Breakthroughs in Science and Technology.

Concerned about the social impact of AI, Bengio helped draft the Montréal Declaration for the Responsible Development of Artificial Intelligence and continues to raise awareness about the importance of mitigating the potentially catastrophic risks associated with future AI systems.

Current Students

Jamal Abou Haibeh

Collaborating Alumni - McGill University

Berkes Anaïs

Collaborating researcher - Cambridge University

Principal supervisor :

berkes.anais@mila.quebec

Rim Assouel

PhD - Université de Montréal

assouelr@mila.quebec

Shahana Chatterjee

Collaborating researcher - N/A

Principal supervisor :

shahana.chatterjee@mila.quebec

sanghyeok.choi@mila.quebec

Xiaoyin Chen

PhD - Université de Montréal

xiaoyin.chen@mila.quebec

Sanghyeok Choi

Collaborating researcher - KAIST

PhD - Université de Montréal

aniket.didolkar@mila.quebec

PhD - Université de Montréal

Co-supervisor :

Guillaume Lajoie

eric.elmoznino@mila.quebec

PhD - Université de Montréal

lena-nehale.ezzine@mila.quebec

Jean-Pierre Falet

PhD - Université de Montréal

jean-pierre.falet@mila.quebec

PhD

pietro.greiner@mila.quebec

Mohsin Hasan

PhD - Université de Montréal

mohsin.hasan@mila.quebec

thomas.jiralerspong@mila.quebec

Moksh Jain

PhD - Université de Montréal

moksh.jain@mila.quebec

PhD - Université de Montréal

Principal supervisor :

Guillaume Lajoie

Collaborating Alumni - Université de Montréal

younesse.kaddar@mila.quebec

Hyeonah Kim

Postdoctorate - Université de Montréal

Principal supervisor :

Alex Hernández-García

hyeonah.kim@mila.quebec

Minsu Kim

Collaborating researcher - Université de Montréal

minsu.kim@mila.quebec

Research Intern - KOREA ADVANCED INSTITUTE OF SCIENCE AND TECHNOLOGY

jeongjae.lee@mila.quebec

cristian-dragos.manta@mila.quebec

Song LIU

Collaborating researcher - s.o.

Collaborating researcher

Cristian Dragos Manta

PhD - Université de Montréal

Co-supervisor :

Dhanya Sridhar

Sarthak Mittal

PhD - Université de Montréal

Principal supervisor :

Independent visiting researcher - Université de Montréal

nguyenph@mila.quebec

Ali Parviz

Collaborating researcher - Ying Wu Coll of Computing

ali.parviz@mila.quebec

Lena Podina

Collaborating researcher - University of Waterloo

Principal supervisor :

elena.podina@mila.quebec

jarrid.rector-brooks@mila.quebec

Jarrid Rector-Brooks

PhD - Université de Montréal

Danyal REHMAN

Postdoctorate - Université de Montréal

danyal.rehman@mila.quebec

oliver.richardson@mila.quebec

Oli RICHARDSON

Postdoctorate - Université de Montréal

Camille Rochefort-Boulanger

PhD - Université de Montréal

Principal supervisor :

Postdoctorate - Université de Montréal

Co-supervisor :

Alex Hernández-García

divya.sharma@mila.quebec

Mélisande Astrid Crystal Teng

Collaborating Alumni - Université de Montréal

Co-supervisor :

Alex Tong

Collaborating Alumni - Université de Montréal

alexander.tong@mila.quebec

Collaborating researcher - University of Cambridge

oszkar.urban@mila.quebec

Donna Vakalis

Collaborating Alumni - Université de Montréal

Co-supervisor :

donna.vakalis@mila.quebec

PhD - Université de Montréal

Principal supervisor :

Glen Berseth

siddarth.venkatraman@mila.quebec

Collaborating researcher - Université de Montréal

esmeralda.whitammer@mila.quebec

Collaborating Alumni

omar.younis@mila.quebec

Collaborating researcher - Université de Montréal

taeyoung.yun@mila.quebec

Skipper: Combining Spatial and Temporal Abstraction for Better Generalization

Tianyu Zhang

PhD - Université de Montréal

tianyu.zhang@mila.quebec

PhD - McGill University

Principal supervisor :

Blog Posts

Generic thumbnail for Mila Blog articles.

February 22, 2024

Mingde Harry Zhao

Safa Alver

Harm van Seijen

Romain Laroche

Doina Precup

Yoshua Bengio

Scaling in the Service of Reasoning & Model-Based ML

April 4, 2023

Yoshua Bengio

Edward J. Hu

A collaboration between Mila and Relation Therapeutics to discover novel synergistic combinations of drugs in vitro

March 23, 2022

Paul Bertin

Jake P. Taylor-King

Yoshua Bengio

March 15, 2022

Generative Flow Networks

Yoshua Bengio

Publications

Inductive Biases for Relational Tasks

Blake Aaron Richards

Current deep learning approaches have shown good in-distribution performance but struggle in out-of-distribution settings. This is especiall… (see more)y true in the case of tasks involving abstract relations like recognizing rules in sequences, as required in many intelligence tests. In contrast, our brains are remarkably flexible at such tasks, an attribute that is likely linked to anatomical constraints on computations. Inspired by this, recent work has explored how enforcing that relational representations remain distinct from sensory representations can help artificial systems. Building on this work, we further explore and formalize the advantages afforded by ``partitioned'' representations of relations and sensory details. We investigate inductive biases that ensure abstract relations are learned and represented distinctly from sensory data across several neural network architectures and show that they outperform existing architectures on out-of-distribution generalization for various relational tasks. These results show that partitioning relational representations from other information streams may be a simple way to augment existing network architectures' robustness when performing relational computations.

2022-03-24

ICLR.cc/2022/Workshop/OSC (poster)

openreview.net

Object-centric Compositional Imagination for Visual Abstract Reasoning

Pau Rodríguez

Like humans devoid of imagination, current machine learning systems lack the ability to adapt to new, unexpected situations by foreseeing th… (see more)em, which makes them unable to solve new tasks by analogical reasoning. In this work, we introduce a new compositional imagination framework that improves a model's ability to generalize. One of the key components of our framework is object-centric inductive biases that enables models to perceive the environment as a series of objects, properties, and transformations. By composing these key ingredients, it is possible to generate new unseen tasks that, when used to train the model, improve generalization. Experiments on a simplified version of the Abstraction and Reasoning Corpus (ARC) demonstrate the effectiveness of our framework.

2022-03-24

ICLR.cc/2022/Workshop/OSC (poster)

openreview.net

A New Era: Intelligent Tutoring Systems Will Transform Online Learning for Millions

Francois St-Hilaire

Dung D. Vu

Antoine Frau

Nathan J. Burns

Farid Faraji

Joseph Potochny

Stephane Robert

Arnaud Roussel

Selene Zheng

Taylor Glazier

Junfel Vincent Romano

Robert Belfer

Muhammad Shayan

Ariella Smofsky

Tommy Delarosbil

Seulmin Ahn

Simon Eden-Walker

Kritika Sony

Ansona Onyi Ching

Sabina Elkins … (see 11 more)

A. Stepanyan

Adela Matajova

Victor Chen

Hossein Sahraei

Robert Larson

N. Markova

Andrew Barkett

Laurent Charlin

Iulian V. Serban

Ekaterina Kochmar

2022-03-02

ArXiv (preprint)

doi.org

arxiv.org

RECOVER: sequential model optimization platform for combination drug repurposing identifies novel synergistic compounds in vitro

Paul Bertin

Jarrid Rector-Brooks

Deepak Sharma

Thomas Gaudelet

Andrew Anighoro

Torsten Gross

Francisco Martínez-Peña

Eileen L. Tang

Suraj M S

Cristian Regep

Jeremy B. R. Hayter

Maksym Korablyov

Nicholas Valiante

Almer Van Der Sloot

Mike Tyers

Charles Roberts

Michael M. Bronstein

Luke L. Lairson

Jake P. Taylor-King

2022-02-06

arXiv (preprint)

doi.org

arxiv.org

Unifying Likelihood-Free Inference with Black-Box Optimization and Beyond

Black-box optimization formulations for biological sequence design have drawn recent attention due to their promising potential impact on th… (see more)e pharmaceutical industry. In this work, we propose to unify two seemingly distinct worlds: likelihood-free inference and black-box optimization, under one probabilistic framework. In tandem, we provide a recipe for constructing various sequence design methods based on this framework. We show how previous optimization approaches can be "reinvented" in our framework, and further propose new probabilistic black-box optimization algorithms. Extensive experiments on sequence design application illustrate the benefits of the proposed methodology.

2022-01-27

International Conference on Learning Representations (spotlight)

doi.org

openreview.net

Boosting Exploration in Multi-Task Reinforcement Learning using Adversarial Networks

Ramnath Kumar

Tristan Deleu

2022-01-26

ArXiv (preprint)

arxiv.org

Biasly: a machine learning based platform for automatic racial discrimination detection in online texts

David Bamman

Chris Dyer

Noah A. Smith. 2014

Steven Bird

Ewan Klein

Edward Loper

Nat-527

Jacob Devlin

Ming-Wei Chang

Kenton Lee

Kristina Toutanova. 2019

Bert

Samuel Gehman

Suchin Gururangan

Maarten Sap

Dan Hendrycks

Kevin Gimpel. 2020

Gaussian

Alex Lamb

Di He … (see 22 more)

Anirudh Goyal

Guolin Ke

Feng-Ju Liao

Mirco Ravanaelli

Zhenzhong Lan

Mingda Chen

Sebastian Goodman

Yann Lecun

Bernhard E. Boser

J. Denker

Don-608 nie Henderson

Robin Howard

Wayne Hubbard

Yinhan Liu

Myle Ott

Naman Goyal

Jingfei Du

Mandar Joshi

Danqi Chen

Omer Levy

Mike Lewis

Warning : this paper contains content that may 001 be offensive or upsetting. 002 Detecting hateful, toxic, and otherwise racist 003 or sexi… (see more)st language in user-generated online con-004 tents has become an increasingly important task 005 in recent years. Indeed, the anonymity, the 006 transience, the size of messages, and the dif-007 ficulty of management, facilitate the diffusion 008 of racist or hateful messages across the Inter-009 net. The critical influence of this cyber-racism 010 is no longer limited to social media, but also 011 has a significant effect on our society : corpo-012 rate business operation, users’ health, crimes, 013 etc. Traditional racist speech reporting chan-014 nels have proven inadequate due to the enor-015 mous explosion of information, so there is an 016 urgent need for a method to automatically and 017 promptly detect texts with racial discrimination. 018 We propose in this work, a machine learning-019 based approach to enable automatic detection 020 of racist text content over the internet. State-of-021 the-art machine learning models that are able 022 to grasp language structures are adapted in this 023 study. Our main contribution include 1) a large 024 scale racial discrimination data set collected 025 from three distinct sources and annotated ac-026 cording to a guideline developed by specialists, 027 2) a set of machine learning models with vari-028 ous architectures for racial discrimination de-029 tection, and 3) a web-browser-based software 030 that assist users to debias their texts when us-031 ing the internet. All these resources are made 032 publicly available.

2021-12-31

(published)

Contrastive introspection (ConSpec) to rapidly identify invariant prototypes for success in RL

Chen Sun

Mila

Wannan Yang

Benjamin Alsbury-Nealy

Thomas Jiralerspong

†. BlakeRichards

Reinforcement learning (RL) algorithms have achieved notable success in recent years, but still struggle with fundamental issues in long-ter… (see more)m credit assignment. It remains diﬃcult to learn in situations where success is contingent upon multiple critical steps that are distant in time from each other and from a sparse reward; as is often the case in real life. Moreover, how RL algorithms assign credit in these diﬃcult situations is typically not coded in a way that can rapidly generalize to new situations. Here, we present an approach using oﬄine contrastive learning, which we call contrastive introspection (ConSpec), that can be added to any existing RL algorithm and addresses both issues. In ConSpec, a contrastive loss is used during oﬄine replay to identify invariances among successful episodes. This takes advantage of the fact that it is easier to retrospectively identify the small set of steps that success is contingent upon than it is to prospectively predict reward at every step taken in the environment. ConSpec stores this knowledge in a collection of prototypes summarizing the intermediate states required for success. During training, arrival at any state that matches these prototypes generates an intrinsic reward that is added to any external rewards. As well, the reward shaping provided by ConSpec can be made to preserve the optimal policy of the underlying RL agent. The prototypes in ConSpec provide two key beneﬁts for credit assignment: (1) They enable rapid identiﬁcation of all the critical states. (2) They do so in a readily interpretable manner, enabling out of distribution generalization when sensory features are altered. In summary, ConSpec is a modular system that can be added to any existing RL algorithm to improve its long-term credit assignment.

2021-12-31

(published)

Enhanced Biomedical Knowledge Discovery From Unstructured Text Using Contextual Embeddings

Iz Beltagy

Kyle Lo

Arman Cohan. 2019

Scib-500

R´ejean Ducharme

P Vincent

Rishi Bommasani

Kelly Davis

Claire Cardie

Billy Chiu

Sampo Pyysalo

Ivan Vuli´c

Extracting knowledge from large, unstruc-001 tured text corpora presents a challenge. Re-002 cently, authors have utilized unsupervised, 003… (see more) static word embeddings to uncover "latent 004 knowledge" contained within domain-speciﬁc 005 scientiﬁc corpora. Here semantic-similarity 006 measures between representations of concepts, 007 objects or entities were used to predict re-008 lationships, which were later veriﬁed using 009 physical methods. Static language models 010 have recently been surpassed at most down-011 stream tasks by massively pre-trained, contex-012 tual language models like BERT. Some have 013 postulated that contextualized embeddings po-014 tentially yield word representations superior 015 to static ones for knowledge-discovery pur-016 poses. In an effort to address this ques-017 tion, two biomedically-trained BERT models 018 (BioBERT, SciBERT) were used to encode 019 n = 500, 1000 or 5000 sentences containing 020 words of interest extracted from a biomedical 021 corpus (Coronavirus Open Research Dataset). 022 The n representations for the words of inter-023 est were subsequently extracted and then ag-024 gregated to yield static-equivalent word rep-025 resentations. These words belonged to the 026 vocabularies of intrinsic benchmarking tools 027 for the biomedical domain (Bio-SimVerb and 028 Bio-SimLex), which assess quality of word 029 representations using semantic-similarity and 030 relatedness measures. Using intrinsic bench-031 marking tasks, feasibility of using contextual-032 ized word representations for knowledge dis-033 covery tasks can be assessed: Word represen-034 tations that better encode described reality are 035 expected to perform better (i.e. closer to do-036 main experts). As postulated, BERT embed-037 dings outperform static counterparts

2021-12-31

(published)

Extended Abstract Track

Amin Mansouri

Jason Hartford

Kartik Ahuja

Christian Shewmake

Simone Azeglio

Arianna Di Bernardo

Nina Miolane

2021-12-31

(published)

S5 Framework: A Review of Self-Supervised Shared Semantic Space Optimization for Multimodal Zero-Shot Learning

Clst

Yonatan Bisk

Ari Holtzman

Jesse Thomason

Ja-740 cob

Joyce Chai

Angeliki Lapata

Jonathan Lazaridou

Alek-742 May

Nicolas sandr Nisnevich

P. PintoJoseph

Turian

Ting Chen

Simon Kornblith

Mohammad Norouzi

Yen-Chun Chen

Linjie Li

Licheng Yu

Ahmed El … (see 89 more)

Faisal Kholy

Zhe Ahmed

Yu Gan

Cheng

Zihan Dai

Hanxiao Liu

Quoc V. Le

Jia Deng

Wei Dong

Richard Socher

Li-Jia Li

K. Liu

Jacob Devlin

Ming-Wei Chang

Kenton Lee

Jesse Dodge

Maarten Sap

Ana Marasovic

Gabriel Agnew

Dirk Ilharco

Groeneveld Matt

Li Dong

Nan Yang

Wenhui Wang

Furu Wei

Yang Liu

Jianfeng Wang

Ming Gao

Zhou

Xiaoyi Dong

Jia Bao

Ting Zhang

Dongdong

Weiming Chen

Lu Zhang

Dong Yuan

Fang Chen

Da-cheng Juan

Chuntian Lu

Zhen Li

Futang Peng

Aleksei Timofeev

Yi-Ting Chen

Yaxi Gao

Tom

Andrew Duerig

Tomkins Sujith

Ravi

Lukasz Kaiser

Aidan N. Gomez

Noam M. Shazeer

Niki Vaswani

Llion Parmar

Jones Jakob

Uszko-850

Alex G. Kendall

Yarin Gal

Roberto Cipolla

Salman H. Khan

Muzammal Naseer

Munawar Hayat

Waqas Zamir

Fahad Shahbaz

Khan

Ranjay Krishna

Yuke Zhu

Oliver Groth

Justin John-867

Kenji Hata

Joshua Kravitz

Stephanie Chen

Mike Lewis

Yinhan Liu

Marjan Naman Goyal

Abdelrahman Ghazvininejad

Omer Mohamed

Levy

Luke Zettlemoyer

Bohan Li

Hao Zhou

Jun-Tao He

Mingxuan Wang

Liunian Harold

Mark Li

Da Yatskar

Yin

Cho-Jui

Kai-Wei Chang

Visualbert

In this review, we aim to inspire research into 001 S elf-S upervised S hared S emantic S pace ( S5 ) 002 multimodal learning problems. We e… (see more)quip non-003 expert researchers with a framework of in-004 formed modeling decisions via an extensive 005 literature review, an actionable modeling check-006 list, as well as a series of novel zero-shot eval-007 uation tasks. The core idea for our S5 check-008 list lies in learning contextual multimodal in-009 teractions at various granularity levels via a 010 shared Transformer encoder with a denoising 011 loss term, which is also regularized by a con-012 trastive loss term to induce a semantic align-013 ment prior on the contextual embedding space. 014 Essentially, we aim to model human concept 015 understanding and thus learn to “put a name to 016 a face”. This ultimately enables interpretable 017 zero-shot S5 generalization on a variety of 018 novel downstream tasks. In summary, this re-019 view provides sufficient background and ac-020 tionable strategies for training cutting-edge S5 021 multimodal networks. 022

2021-12-31

(published)

Harvesting Mature Relation Extraction Models from Limited Seed Knowledge: A Self-Development Framework for DS Rule Expansion

Raphael Hoffmann

Congle Zhang

Xiao Ling

Yankai Lin

Shiqi Shen

Zhiyuan Liu

Huanbo Luan

Christopher D Manning

M. Surdeanu

John Bauer

Adriana Romero

Pietro Lio’