Yoshua Bengio

jamal.abouhaibeh@mila.quebec

Biographie

*Pour toute demande média, veuillez écrire à medias@mila.quebec.

Pour plus d’information, contactez Cassidy MacNeil, adjointe principale et responsable des opérations cassidy.macneil@mila.quebec.

Reconnu comme une sommité mondiale en intelligence artificielle, Yoshua Bengio s’est surtout distingué par son rôle de pionnier en apprentissage profond, ce qui lui a valu le prix A. M. Turing 2018, le « prix Nobel de l’informatique », avec Geoffrey Hinton et Yann LeCun. Il est professeur titulaire à l’Université de Montréal, fondateur et conseiller scientifique de Mila – Institut québécois d’intelligence artificielle, et codirige en tant que senior fellow le programme Apprentissage automatique, apprentissage biologique de l'Institut canadien de recherches avancées (CIFAR). Il occupe également la fonction de conseiller spécial et directeur scientifique fondateur d’IVADO.

En 2018, il a été l’informaticien qui a recueilli le plus grand nombre de nouvelles citations au monde. En 2019, il s’est vu décerner le prestigieux prix Killam. Depuis 2022, il détient le plus grand facteur d’impact (h-index) en informatique à l’échelle mondiale. Il est fellow de la Royal Society de Londres et de la Société royale du Canada, et officier de l’Ordre du Canada.

Soucieux des répercussions sociales de l’IA et de l’objectif que l’IA bénéficie à tous, il a contribué activement à la Déclaration de Montréal pour un développement responsable de l’intelligence artificielle.

Étudiants actuels

Jamal Abou Haibeh

Collaborateur·rice alumni - McGill

Collaborateur·rice de recherche - Cambridge University

Superviseur⋅e principal⋅e :

David Rolnick

berkes.anais@mila.quebec

sanghyeok.choi@mila.quebec

Rim Assouel

Doctorat - UdeM

assouelr@mila.quebec

Xiaoyin Chen

Doctorat - UdeM

xiaoyin.chen@mila.quebec

Sanghyeok Choi

Collaborateur·rice de recherche - KAIST

Doctorat - UdeM

aniket.didolkar@mila.quebec

Doctorat - UdeM

Co-superviseur⋅e :

Guillaume Lajoie

eric.elmoznino@mila.quebec

Doctorat - UdeM

lena-nehale.ezzine@mila.quebec

Jean-Pierre Falet

Doctorat - UdeM

jean-pierre.falet@mila.quebec

Doctorat

pietro.greiner@mila.quebec

Mohsin Hasan

Doctorat - UdeM

mohsin.hasan@mila.quebec

thomas.jiralerspong@mila.quebec

Moksh Jain

Doctorat - UdeM

moksh.jain@mila.quebec

Doctorat - UdeM

Superviseur⋅e principal⋅e :

Guillaume Lajoie

Collaborateur·rice alumni - UdeM

younesse.kaddar@mila.quebec

Minsu Kim

Collaborateur·rice de recherche - UdeM

minsu.kim@mila.quebec

Hyeonah Kim

Postdoctorat - UdeM

Superviseur⋅e principal⋅e :

Alex Hernández-García

hyeonah.kim@mila.quebec

Jeongjae Lee

Collaborateur·rice de recherche - KOREA ADVANCED INSTITUTE OF SCIENCE AND TECHNOLOGY

jeongjae.lee@mila.quebec

cristian-dragos.manta@mila.quebec

Song LIU

Collaborateur·rice de recherche - s.o.

Collaborateur·rice de recherche

Cristian Dragos Manta

Doctorat - UdeM

Co-superviseur⋅e :

Dhanya Sridhar

Sarthak Mittal

Doctorat - UdeM

Superviseur⋅e principal⋅e :

Visiteur de recherche indépendant - UdeM

nguyenph@mila.quebec

Ali Parviz

Collaborateur·rice de recherche - Ying Wu Coll of Computing

ali.parviz@mila.quebec

Lena Podina

Collaborateur·rice de recherche - University of Waterloo

Superviseur⋅e principal⋅e :

David Rolnick

elena.podina@mila.quebec

jarrid.rector-brooks@mila.quebec

Jarrid Rector-Brooks

Doctorat - UdeM

Danyal REHMAN

Postdoctorat - UdeM

danyal.rehman@mila.quebec

oliver.richardson@mila.quebec

Oli RICHARDSON

Postdoctorat - UdeM

Camille Rochefort-Boulanger

Doctorat - UdeM

Superviseur⋅e principal⋅e :

Postdoctorat - UdeM

Co-superviseur⋅e :

Alex Hernández-García

divya.sharma@mila.quebec

Mélisande Astrid Crystal Teng

Collaborateur·rice alumni - UdeM

Co-superviseur⋅e :

Alex Tong

Collaborateur·rice alumni - UdeM

alexander.tong@mila.quebec

Collaborateur·rice de recherche - University of Cambridge

oszkar.urban@mila.quebec

donna.vakalis@mila.quebec

Donna Vakalis

Collaborateur·rice alumni - UdeM

Co-superviseur⋅e :

David Rolnick

Doctorat - UdeM

Superviseur⋅e principal⋅e :

Glen Berseth

siddarth.venkatraman@mila.quebec

Collaborateur·rice de recherche

joseph.viviano2@mila.quebec

Collaborateur·rice de recherche - UdeM

esmeralda.whitammer@mila.quebec

Collaborateur·rice alumni

omar.younis@mila.quebec

Collaborateur·rice de recherche - UdeM

taeyoung.yun@mila.quebec

Skipper : combiner l’abstraction spatiale et temporelle afin d’améliorer la généralisation

Tianyu Zhang

Doctorat - UdeM

tianyu.zhang@mila.quebec

Doctorat - McGill

Superviseur⋅e principal⋅e :

Harry Zhao

Collaborateur·rice alumni - McGill

Superviseur⋅e principal⋅e :

Billets de blogue

Generic thumbnail for Mila Blog articles.

22 février 2024

par

Mingde Harry Zhao

Safa Alver

Harm van Seijen

Romain Laroche

Doina Precup

Yoshua Bengio

Mise à l’échelle au service du raisonnement et de l’apprentissage automatique basé sur un modèle

Scaling in the service of reasoning & model-based ML

4 avril 2023

par

Yoshua Bengio

Edward J. Hu

Une collaboration entre Mila et Relation Therapeutics pour découvrir in vitro de nouvelles associations médicamenteuses synergiques

A collaboration between Mila and Relation Therapeutics to discover novel synergistic combinations of drugs in vitro

23 mars 2022

par

Paul Bertin

Jake P. Taylor-King

Yoshua Bengio

Les réseaux de flot génératifs

15 mars 2022

par

Yoshua Bengio

Publications

Enhanced Biomedical Knowledge Discovery From Unstructured Text Using Contextual Embeddings

Iz Beltagy

Kyle Lo

Arman Cohan. 2019

Scib-500

R´ejean Ducharme

P Vincent

Rishi Bommasani

Kelly Davis

Claire Cardie

Billy Chiu

Sampo Pyysalo

Ivan Vuli´c

Extracting knowledge from large, unstruc-001 tured text corpora presents a challenge. Re-002 cently, authors have utilized unsupervised, 003… (voir plus) static word embeddings to uncover "latent 004 knowledge" contained within domain-speciﬁc 005 scientiﬁc corpora. Here semantic-similarity 006 measures between representations of concepts, 007 objects or entities were used to predict re-008 lationships, which were later veriﬁed using 009 physical methods. Static language models 010 have recently been surpassed at most down-011 stream tasks by massively pre-trained, contex-012 tual language models like BERT. Some have 013 postulated that contextualized embeddings po-014 tentially yield word representations superior 015 to static ones for knowledge-discovery pur-016 poses. In an effort to address this ques-017 tion, two biomedically-trained BERT models 018 (BioBERT, SciBERT) were used to encode 019 n = 500, 1000 or 5000 sentences containing 020 words of interest extracted from a biomedical 021 corpus (Coronavirus Open Research Dataset). 022 The n representations for the words of inter-023 est were subsequently extracted and then ag-024 gregated to yield static-equivalent word rep-025 resentations. These words belonged to the 026 vocabularies of intrinsic benchmarking tools 027 for the biomedical domain (Bio-SimVerb and 028 Bio-SimLex), which assess quality of word 029 representations using semantic-similarity and 030 relatedness measures. Using intrinsic bench-031 marking tasks, feasibility of using contextual-032 ized word representations for knowledge dis-033 covery tasks can be assessed: Word represen-034 tations that better encode described reality are 035 expected to perform better (i.e. closer to do-036 main experts). As postulated, BERT embed-037 dings outperform static counterparts

2021-12-31

(publié)

Extended Abstract Track

Amin Mansouri

Jason Hartford

Kartik Ahuja

Christian Shewmake

Simone Azeglio

Arianna Di Bernardo

Nina Miolane

2021-12-31

(publié)

S5 Framework: A Review of Self-Supervised Shared Semantic Space Optimization for Multimodal Zero-Shot Learning

Clst

Yonatan Bisk

Ari Holtzman

Jesse Thomason

Ja-740 cob

Joyce Chai

Angeliki Lapata

Jonathan Lazaridou

Alek-742 May

Nicolas sandr Nisnevich

P. PintoJoseph

Turian

Ting Chen

Simon Kornblith

Mohammad Norouzi

Yen-Chun Chen

Linjie Li

Licheng Yu

Ahmed El … (voir 89 de plus)

Faisal Kholy

Zhe Ahmed

Yu Gan

Cheng

Zihan Dai

Hanxiao Liu

Quoc V. Le

Jia Deng

Wei Dong

Richard Socher

Li-Jia Li

K. Liu

Jacob Devlin

Ming-Wei Chang

Kenton Lee

Jesse Dodge

Maarten Sap

Ana Marasovic

Gabriel Agnew

Dirk Ilharco

Groeneveld Matt

Li Dong

Nan Yang

Wenhui Wang

Furu Wei

Yang Liu

Jianfeng Wang

Ming Gao

Zhou

Xiaoyi Dong

Jia Bao

Ting Zhang

Dongdong

Weiming Chen

Lu Zhang

Dong Yuan

Fang Chen

Da-cheng Juan

Chuntian Lu

Zhen Li

Futang Peng

Aleksei Timofeev

Yi-Ting Chen

Yaxi Gao

Tom

Andrew Duerig

Tomkins Sujith

Ravi

Lukasz Kaiser

Aidan N. Gomez

Noam M. Shazeer

Niki Vaswani

Llion Parmar

Jones Jakob

Uszko-850

Alex G. Kendall

Yarin Gal

Roberto Cipolla

Salman H. Khan

Muzammal Naseer

Munawar Hayat

Waqas Zamir

Fahad Shahbaz

Khan

Ranjay Krishna

Yuke Zhu

Oliver Groth

Justin John-867

Kenji Hata

Joshua Kravitz

Stephanie Chen

Mike Lewis

Yinhan Liu

Marjan Naman Goyal

Abdelrahman Ghazvininejad

Omer Mohamed

Levy

Luke Zettlemoyer

Bohan Li

Hao Zhou

Jun-Tao He

Mingxuan Wang

Liunian Harold

Mark Li

Da Yatskar

Yin

Cho-Jui

Kai-Wei Chang

Visualbert

In this review, we aim to inspire research into 001 S elf-S upervised S hared S emantic S pace ( S5 ) 002 multimodal learning problems. We e… (voir plus)quip non-003 expert researchers with a framework of in-004 formed modeling decisions via an extensive 005 literature review, an actionable modeling check-006 list, as well as a series of novel zero-shot eval-007 uation tasks. The core idea for our S5 check-008 list lies in learning contextual multimodal in-009 teractions at various granularity levels via a 010 shared Transformer encoder with a denoising 011 loss term, which is also regularized by a con-012 trastive loss term to induce a semantic align-013 ment prior on the contextual embedding space. 014 Essentially, we aim to model human concept 015 understanding and thus learn to “put a name to 016 a face”. This ultimately enables interpretable 017 zero-shot S5 generalization on a variety of 018 novel downstream tasks. In summary, this re-019 view provides sufficient background and ac-020 tionable strategies for training cutting-edge S5 021 multimodal networks. 022

2021-12-31

(publié)

Harvesting Mature Relation Extraction Models from Limited Seed Knowledge: A Self-Development Framework for DS Rule Expansion

Raphael Hoffmann

Congle Zhang

Xiao Ling

Yankai Lin

Shiqi Shen

Zhiyuan Liu

Huanbo Luan

Christopher D Manning

M. Surdeanu

John Bauer

Adriana Romero

Pietro Lio’

Xuanhui Wang

Cheng Li

Nadav Golbandi

Bendersky Marc

Najork. 2018

The

Wentao Wu … (voir 2 de plus)

Hongsong Li

Haixun Wang

Distantly-supervised relation extraction 001 (DSRE) is an effective method to scale relation 002 extraction (RE) to large unlabeled corpora … (voir plus)003 with the utilization of knowledge bases (KBs), 004 but suffers from the scale of KBs and the 005 introduced noise. 006 To alleviate the above two problems, we 007 propose a novel framework called S elf-008 devel O pment r U le ex P ansion ( SOUP ), which 009 starts from limited amount of labeled data 010 and continuously produces low-noise labels on 011 large-scaled unlabeled data by a growing learn-012 able logical rules set. 013 Specifically, SOUP achieves a mutual enhance-014 ment of RE model and logical rules set, first 015 a RE model is trained on the labeled data to 016 summarize the knowledge, then the knowledge 017 is utilized to explore candidate rules from unla-018 beled data, finally high-quality candidates are 019 selected in a graph-based ranking manner to ex-020 tend the logical rules set and new rule-labeled 021 data are provided for better RE model training. 022 Experiments on wiki20 dataset demonstrate 023 that, with limited seed knowledge from small-024 scaled manually labeled data, SOUP achieves 025 significant improvement compared to baselines 026 by producing continuous growth of both logical 027 rules and the RE model, and that labeling noise 028 of SOUP is much less than DS. Furthermore, 029 RE model enhanced by SOUP with 1.6k logical 030 rules learned from prior knowledge could pro-031 duce an equivalent performance to the model 032 trained on data labeled in DS manner by 72k 033 relational facts of KBs. 034

2021-12-31

(publié)

How Transferable Are Features in Convolutional Neural Network Acoustic Models across Languages?

Jessica A.F. Thompson

Marc Schönwiesner

Daniel Willett

Characterization of the representations learned in intermediate layers of deep networks can provide valuable insight into the nature of a ta… (voir plus)sk and can guide the development of well-tailored learning strategies. Here we study convolutional neural network (CNN)-based acoustic models in the context of automatic speech recognition. Adapting a method proposed by [1], we measure the transferability of each layer between English, Dutch and German to assess their language-specificity. We observed three distinct regions of transferability: (1) the first two layers were entirely transferable between languages, (2) layers 2–8 were also highly transferable but we found some evidence of language specificity, (3) the subsequent fully connected layers were more language specific but could be successfully finetuned to the target language. To further probe the effect of weight freezing, we performed follow-up experiments using freeze-training [2]. Our results are consistent with the observation that CNNs converge ‘bottom up’ during training and demonstrate the benefit of freeze training, especially for transfer learning.

2021-12-31

IEEE International Conference on Acoustics, Speech and Signal Processing (inconnu)

Interpolation Consistency Training for Semi-Supervised Learning

Juho Kannala

David Lopez-Paz

Arno Solin

2021-12-31

Neural Networks (publié)

PMFL: Partial Meta-Federated Learning for heterogeneous tasks and its applications on real-world medical records

Tianyi Zhang

Shirui Zhang

Ziwei Chen

Dianbo Liu

Federated machine learning is a versatile and flexible tool to utilize distributed data from different sources, especially when communicatio… (voir plus)n technology develops rapidly and an unprecedented amount of data could be collected on mobile devices nowadays. Federated learning method exploits not only the data but the computational power of all devices in the network to achieve more efficient model training. Nevertheless, while most traditional federated learning methods work well for homogeneous data and tasks, adapting the method to heterogeneous data and task distribution is challenging. This limitation has constrained the applications of federated learning in real-world contexts, especially in healthcare settings. Inspired by the fundamental idea of meta-learning, in this study we propose a new algorithm, which is an integration of federated learning and meta-learning, to tackle this issue. In addition, owing to the advantage of transfer learning for model generalization, we further improve our algorithm by introducing partial parameter sharing to balance global and local learning. We name this method partial meta-federated learning (PMFL). Finally, we apply the algorithms to two medical datasets. We show that our algorithm could obtain the fastest training speed and achieve the best performance when dealing with heterogeneous medical datasets.

2021-12-31

IEEE Conference on Big Data (publié)

Predicting Tactical Solutions to Operational Planning Problems under Imperfect Information

Eric Larsen

Andrea Lodi

2021-12-31

INFORMS Journal on Computing (publié)

(Private)-Retroactive Carbon Pricing [(P)ReCaP]: A Market-based Approach for Climate Finance and Risk Assessment

Prateek Gupta

Dylan Radovic

Maarten Scholl

Andrew Williams

Christian Schroeder de Witt

Tianyu Zhang

Yang Zhang

Insufficient Social Cost of Carbon (SCC) estimation methods and short-term decision-making horizons have hindered the ability of carbon emit… (voir plus)ters to properly correct for the negative externalities of climate change, as well as the capacity of nations to balance economic and climate policy. To overcome these limitations, we introduce Retrospective Social Cost of Carbon Updating (ReSCCU), a novel mechanism that corrects for these limitations as empirically measured evidence is collected. To implement ReSCCU in the context of carbon taxation, we propose Retroactive Carbon Pricing (ReCaP), a market mechanism in which polluters offload the payment of ReSCCU adjustments to insurers. To alleviate systematic risks and minimize government involvement, we introduce the Private ReCaP (PReCaP) prediction market, which could see real-world implementation based on the engagement of a few high net-worth individuals or independent institutions.

2021-12-31

SSRN Electronic Journal (inconnu)

Straight to the Tree: Constituency Parsing with Neural Syntactic Distance

In this work, we propose a novel constituency parsing scheme. The model predicts a vector of real-valued scalars, named syntactic distances,… (voir plus) for each split position in the input sentence. The syntactic distances specify the order in which the split points will be selected, recursively partitioning the input, in a top-down fashion. Compared to traditional shift-reduce parsing schemes, our approach is free from the potential problem of compounding errors, while being faster and easier to parallelize. Our model achieves competitive performance amongst single model, discriminative parsers in the PTB dataset and outperforms previous models in the CTB dataset.

2021-12-31

Annual Meeting of the Association for Computational Linguistics (publié)

TaHiD: Tackling Data Hiding in Fake News Detection with News Propagation Networks

Adrien Benamira

Benjamin Devillers

Etienne Lesot

Ayush K. Ray

Manal Saadi

Fragkiskos D 587

Steven Bird

Ewan Klein

Edward Loper

Nat-593

Carlos Castillo

Marcelo Mendoza

Barbara Poblete

Daryna Dementieva

Alexander Panchenko

Jacob Devlin

Ming-Wei Chang

Kenton Lee

Ashish Vaswani

Noam M. Shazeer … (voir 8 de plus)

Niki Parmar

Adriana Romero

Pietro Lio’

Yaqing Wang

Fenglong Ma

Zhiwei Jin

Ye Yuan

Fake news with detrimental societal effects has 001 attracted extensive attention and research. De-002 spite early success, the state-of-the… (voir plus)-art meth-003 ods fall short of considering the propagation 004 of news. News propagates at different times 005 through different mediums, including users, 006 comments, and sources, which form the news 007 propagation network. Moreover, the serious 008 problem of data hiding arises, which means 009 that fake news publishers disguise fake news 010 as real to confuse users by deleting comments 011 that refute the rumor or deleting the news itself 012 when it has been spread widely. Existing meth-013 ods do not consider the propagation of news 014 and fail to identify what matters in the process, 015 which leads to fake news hiding in the prop-016 agation network and escaping from detection. 017 Inspired by the propagation of news, we pro-018 pose a novel fake news detection framework 019 named TaHiD, which models the propagation 020 as a heterogeneous dynamic graph and contains 021 the propagation attention module to measure 022 the influence of different propagation. Exper-023 iments demonstrate that TaHiD extracts use-024 ful information from the news propagation net-025 work and outperforms state-of-the-art methods 026 on several benchmark datasets for fake news 027 detection. Additional studies also show that 028 TaHiD is capable of identifying fake news in 029 the case of data hiding. 030

2021-12-31

(publié)

A Two-Stream Continual Learning System With Variational Domain-Agnostic Feature Replay

Qicheng Lao

Xiang Jiang

Mohammad Havaei

Learning in nonstationary environments is one of the biggest challenges in machine learning. Nonstationarity can be caused by either task dr… (voir plus)ift, i.e., the drift in the conditional distribution of labels given the input data, or the domain drift, i.e., the drift in the marginal distribution of the input data. This article aims to tackle this challenge with a modularized two-stream continual learning (CL) system, where the model is required to learn new tasks from a support stream and adapted to new domains in the query stream while maintaining previously learned knowledge. To deal with both drifts within and across the two streams, we propose a variational domain-agnostic feature replay-based approach that decouples the system into three modules: an inference module that filters the input data from the two streams into domain-agnostic representations, a generative module that facilitates the high-level knowledge transfer, and a solver module that applies the filtered and transferable knowledge to solve the queries. We demonstrate the effectiveness of our proposed approach in addressing the two fundamental scenarios and complex scenarios in two-stream CL.

2021-12-31

IEEE Transactions on Neural Networks and Learning Systems (publié)