Doina Precup

Lynn Cherif

Master's Research - McGill University

lynn.cherif@mila.quebec

Ray Chua

PhD - McGill University

Co-supervisor :

Blake Richards

chuaraym@mila.quebec

Wesley Chung

PhD - McGill University

Principal supervisor :

jonathan.colaco-carr@mila.quebec

chungwes@mila.quebec

Jonathan Colaço Carr

Master's Research - McGill University

Principal supervisor :

Prakash Panangaden

Élodie Côté-Gauthier Coté-Gauthier

Research Intern - McGill University

elodie.cote-gauthier@mila.quebec

Nathan de Lara

Research Intern - McGill University

nathan.de-lara@mila.quebec

Manuel Del Verme

PhD - McGill University

delvermm@mila.quebec

Jesse Farebrother

PhD - McGill University

Principal supervisor :

Marc Gendron-Bellemare

PhD - McGill University

Co-supervisor :

Mohammad Sami Nur Islam Islam

fujimots@mila.quebec

Will Hua

Master's Research - McGill University

Co-supervisor :

Guy Wolf

chenqing.hua@mila.quebec

Howard Huang

PhD - McGill University

huanghow@mila.quebec

Haque Ishfaq

PhD - McGill University

PhD - McGill University

Research Intern - McGill University

mohammad-sami-nur.islam@mila.quebec

thomas.jiralerspong@mila.quebec

Arushi Jain

PhD - McGill University

Master's Research - Université de Montréal

Principal supervisor :

Yoshua Bengio

PhD - McGill University

Postdoctorate - McGill University

flemming.kondrup@mila.quebec

Elaine Lau

Master's Research - McGill University

elaine.lau@mila.quebec

Jonathan Lebensold

PhD - McGill University

Sitao Luan

PhD - McGill University

Ray Luo

PhD - McGill University

Principal supervisor :

Xujie Si

luo.ziyan@mila.quebec

G McCracken

PhD - McGill University

mccrackg@mila.quebec

shahrad.mohammadzadeh@mila.quebec

Jaume Minano Masip

PhD - McGill University

masipmij@mila.quebec

Shahrad Mohammadzadeh

Collaborating researcher - McGill University

Principal supervisor :

Reihaneh Rabbany

padideh.nouri@mila.quebec

Gabriela Moisescu-Pareja

Master's Research - McGill University

moisescg@mila.quebec

Girdhar Neil Girdhar

Collaborating researcher - McGill University

neil.girdhar@mila.quebec

Padideh Nouri

Master's Research - Université de Montréal

Sarath Chandar Anbil Parthipan

Charles Onu

PhD - McGill University

PhD - McGill University

Co-supervisor :

Nate Rahn

PhD - McGill University

Principal supervisor :

Marc Gendron-Bellemare

nathan.rahn@mila.quebec

Janarthanan Rajendran

Postdoctorate - Université de Montréal

Principal supervisor :

janarthanan.rajendran@mila.quebec

Sahand Rezaei-Shoshtari

PhD - McGill University

Co-supervisor :

sahand.rezaei-shoshtari@mila.quebec

PhD - McGill University

Co-supervisor :

PhD - McGill University

Nishanth Anand Vemgal

PhD - McGill University

Master's Research - McGill University

nikhil-murali.vemgal@mila.quebec

guangyuan.wang@mila.quebec

David Anthony Venuto

PhD - McGill University

venutoda@mila.quebec

Priyesh Vijayan

PhD - McGill University

Co-supervisor :

Samira Ebrahimi Kahou

Research Intern - McGill University

Keyu Wang Wang

Research Intern - McGill University

keyu.wang@mila.quebec

shuyuan.zhang@mila.quebec

Steve Wen

Undergraduate - McGill University

steve.wen@mila.quebec

Shuyuan Zhang

PhD - McGill University

Harry Zhao

PhD - McGill University

Co-supervisor :

Blog Posts

Generic thumbnail for Mila Blog articles.

February 22, 2024

Skipper: Combining Spatial and Temporal Abstraction for Better Generalization

Mingde Harry Zhao

Safa Alver

Harm van Seijen

Romain Laroche

Doina Precup

Yoshua Bengio

Read the article

Publications

When Do We Need Graph Neural Networks for Node Classification?

Sitao Luan

Chenqing Hua

Qincheng Lu

Jiaqi Zhu

Xiao-Wen Chang

2024-02-20

Complex Networks & Their Applications XII (published)

Gintare Karolina Dziugaite

Mixtures of Experts Unlock Parameter Scaling for Deep RL

Johan Samir Obando Ceron

Ghada Sokar

Timon Willi

Clare Lyle

Jesse Farebrother

Jakob Nicolaus Foerster

Pablo Samuel Castro

2024-02-13

ArXiv (preprint)

On the Privacy of Selection Mechanisms with Gaussian Noise

Jonathan Lebensold

Borja Balle

2024-02-09

ArXiv (preprint)

Effective Protein-Protein Interaction Exploration with PPIretrieval

Chenqing Hua

Connor Coley

Guy Wolf

Shuangjia Zheng

2024-02-06

ArXiv (preprint)

Consciousness-Inspired Spatio-Temporal Abstractions for Better Generalization in Reinforcement Learning

Harry Zhao

Harry Zhao 0001

Mingde Zhao

Safa Alver

Harm van Seijen

Romain Laroche

Yoshua Bengio

Inspired by human conscious planning, we propose Skipper, a model-based reinforcement learning framework utilizing spatio-temporal abstracti… (see more)ons to generalize better in novel situations. It automatically decomposes the given task into smaller, more manageable subtasks, and thus enables sparse decision-making and focused computation on the relevant parts of the environment. The decomposition relies on the extraction of an abstracted proxy problem represented as a directed graph, in which vertices and edges are learned end-to-end from hindsight. Our theoretical analyses provide performance guarantees under appropriate assumptions and establish where our approach is expected to be helpful. Generalization-focused experiments validate Skipper’s significant advantage in zero-shot generalization, compared to some existing state-of-the-art hierarchical planning methods.

2024-01-16

ICLR.cc/2024/Conference (poster)

openreview.net

Provable and Practical: Efficient Exploration in Reinforcement Learning via Langevin Monte Carlo

Haque Ishfaq

Qingfeng Lan

Pan Xu

A. Rupam Mahmood

Animashree Anandkumar

Kamyar Azizzadenesheli

We present a scalable and effective exploration strategy based on Thompson sampling for reinforcement learning (RL). One of the key shortcom… (see more)ings of existing Thompson sampling algorithms is the need to perform a Gaussian approximation of the posterior distribution, which is not a good surrogate in most practical settings. We instead directly sample the Q function from its posterior distribution, by using Langevin Monte Carlo, an efficient type of Markov Chain Monte Carlo (MCMC) method. Our method only needs to perform noisy gradient descent updates to learn the exact posterior distribution of the Q function, which makes our approach easy to deploy in deep RL. We provide a rigorous theoretical analysis for the proposed method and demonstrate that, in the linear Markov decision process (linear MDP) setting, it has a regret bound of

2024-01-16

ICLR.cc/2024/Conference (poster)

openreview.net

Connecting Weighted Automata, Tensor Networks and Recurrent Neural Networks through Spectral Learning

Tianyu Li

Guillaume Rabusseau

2024-01-01

Mach. Learn. (published)

Policy Gradient Methods in the Presence of Symmetries and State Abstractions

Prakash Panangaden

Sahand Rezaei-Shoshtari

Rosie Zhao

Nash Learning from Human Feedback

R'emi Munos

Michal Valko

Daniele Calandriello

M. G. Azar

Mark Rowland

Zhaohan Daniel Guo

Yunhao Tang

Matthieu Geist

Thomas Mesnard

Andrea Michi

Marco Selvi

Sertan Girgin

Nikola Momchev

Olivier Bachem

Daniel J Mankowitz

Bilal Piot

Reinforcement learning from human feedback (RLHF) has emerged as the main paradigm for aligning large language models (LLMs) with human pref… (see more)erences. Typically, RLHF involves the initial step of learning a reward model from human feedback, often expressed as preferences between pairs of text generations produced by a pre-trained LLM. Subsequently, the LLM's policy is fine-tuned by optimizing it to maximize the reward model through a reinforcement learning algorithm. However, an inherent limitation of current reward models is their inability to fully represent the richness of human preferences and their dependency on the sampling distribution. In this study, we introduce an alternative pipeline for the fine-tuning of LLMs using pairwise human feedback. Our approach entails the initial learning of a preference model, which is conditioned on two inputs given a prompt, followed by the pursuit of a policy that consistently generates responses preferred over those generated by any competing policy, thus defining the Nash equilibrium of this preference model. We term this approach Nash learning from human feedback (NLHF). In the context of a tabular policy representation, we present a novel algorithmic solution, Nash-MD, founded on the principles of mirror descent. This algorithm produces a sequence of policies, with the last iteration converging to the regularized Nash equilibrium. Additionally, we explore parametric representations of policies and introduce gradient descent algorithms for deep-learning architectures. To demonstrate the effectiveness of our approach, we present experimental results involving the fine-tuning of a LLM for a text summarization task. We believe NLHF offers a compelling avenue for preference learning and policy optimization with the potential of advancing the field of aligning LLMs with human preferences.

2023-12-01

ArXiv (preprint)

Learning domain-invariant classifiers for infant cry sounds

Charles Onu

Hemanth K. Sheetha

Arsenii Gorin

2023-11-30

ArXiv (preprint)