Aaron Courville

alexandre.ganito@mila.quebec

adrien.taiga@mila.quebec

Alexandre Diz Ganito

Maîtrise recherche - Université de Montréal

Amr Khalifa

Doctorat - Université de Montréal

amr-khalifa.alshaykh@mila.quebec

andrei.nicolicioiu@mila.quebec

Andrei Nicolicioiu

Doctorat - Université de Montréal

ching-lam.choi@mila.quebec

Ankit Vani

Doctorat - Université de Montréal

Baccalauréat - Université de Montréal

Doctorat - Université de Montréal

Superviseur⋅e principal⋅e :

Doctorat - Université de Montréal

Co-superviseur⋅e :

Yoshua Bengio

dinghuai.zhang@mila.quebec

Esra'a Saleh

Doctorat - Université de Montréal

Superviseur⋅e principal⋅e :

Glen Berseth

esraa.saleh@mila.quebec

evgenii.nikishin@mila.quebec

Evgenii Nikishin

Doctorat - Université de Montréal

Superviseur⋅e principal⋅e :

Pierre-Luc Bacon

Faruk Ahmed

Doctorat - Université de Montréal

faruk.ahmed@mila.quebec

Johan Samir Obando Ceron

Doctorat - Université de Montréal

Co-superviseur⋅e :

Pablo Samuel Castro

johan.ceron@mila.quebec

Juan Duque

Doctorat - Université de Montréal

juan.duque@mila.quebec

manoosh.samiei@mila.quebec

Manoosh Samiei

Doctorat - Université de Montréal

Doctorat - Université de Montréal

Co-superviseur⋅e :

mohammed.muqeeth@mila.quebec

schwarzm@mila.quebec

Hattie Zhou

Doctorat - Université de Montréal

Superviseur⋅e principal⋅e :

Hugo Larochelle

mengfei.zhou@mila.quebec

Michael Noukhovitch

Doctorat - Université de Montréal

Collaborateur·rice de recherche

morgane.moss@mila.quebec

Muqeeth Mohammed

Collaborateur·rice de recherche - Université de Montréal

Stagiaire de recherche - Ghent University

pietro.mazzaglia@mila.quebec

Stagiaire de recherche - Université de Montréal

razvan.ciuca@mila.quebec

Rishabh Agarwal

Doctorat - Université de Montréal

Co-superviseur⋅e :

samuel.lavoie@mila.quebec

Samuel Lavoie Marchildon

Doctorat - Université de Montréal

Sarvjeet Ghotra

Maîtrise recherche - Université de Montréal

Superviseur⋅e principal⋅e :

Aishwarya Agrawal

sarvjeet-singh.ghotra@mila.quebec

xiaofeng.zhang@mila.quebec

Arian Hosseini

Doctorat - Université de Montréal

Shawn Tan

Doctorat - Université de Montréal

Doctorat - Université de Montréal

Doctorat - Université de Montréal

Yusong Wu

Doctorat - Université de Montréal

Superviseur⋅e principal⋅e :

Anna (Cheng-Zhi) Huang

wu.yusong@mila.quebec

Zhixuan Lin

Doctorat - Université de Montréal

zhixuan.lin@mila.quebec

Publications

Mastering the Unsupervised Reinforcement Learning Benchmark from Pixels

Sai Rajeswar

Pietro Mazzaglia

Tim Verbelen

Alexandre Piché

Bart Dhoedt

Alexandre Lacoste

Controlling artificial agents from visual sensory data is an arduous task. Reinforcement learning (RL) algorithms can succeed but require la… (voir plus)rge amounts of interactions between the agent and the environment. To alleviate the issue, unsupervised RL proposes to employ self-supervised interaction and learning, for adapting faster to future tasks. Yet, as shown in the Unsupervised RL Benchmark (URLB; Laskin et al. 2021), whether current unsupervised strategies can improve generalization capabilities is still unclear, especially in visual control settings. In this work, we study the URLB and propose a new method to solve it, using unsupervised model-based RL, for pre-training the agent, and a task-aware fine-tuning strategy combined with a new proposed hybrid planner, Dyna-MPC, to adapt the agent for downstream tasks. On URLB, our method obtains 93.59% overall normalized performance, surpassing previous baselines by a staggering margin. The approach is empirically evaluated through a large-scale empirical study, which we use to validate our design choices and analyze our models. We also show robust performance on the Real-Word RL benchmark, hinting at resiliency to environment perturbations during adaptation. Project website: https://masteringurlb.github.io/

2023-04-24

ICML.cc/2023/Conference (poster)

proceedings.mlr.press

Mastering the Unsupervised Reinforcement Learning Benchmark from Pixels

Sai Rajeswar

Pietro Mazzaglia

Tim Verbelen

Alexandre Piché

Bart Dhoedt

Alexandre Lacoste

2023-04-24

ICML.cc/2023/Conference (publié)

Generative Augmented Flow Networks

Ling Pan

Dinghuai Zhang

Longbo Huang

Yoshua Bengio

The Generative Flow Network is a probabilistic framework where an agent learns a stochastic policy for object generation, such that the prob… (voir plus)ability of generating an object is proportional to a given reward function. Its effectiveness has been shown in discovering high-quality and diverse solutions, compared to reward-maximizing reinforcement learning-based methods. Nonetheless, GFlowNets only learn from rewards of the terminal states, which can limit its applicability. Indeed, intermediate rewards play a critical role in learning, for example from intrinsic motivation to provide intermediate feedback even in particularly challenging sparse reward tasks. Inspired by this, we propose Generative Augmented Flow Networks (GAFlowNets), a novel learning framework to incorporate intermediate rewards into GFlowNets. We specify intermediate rewards by intrinsic motivation to tackle the exploration problem in sparse reward environments. GAFlowNets can leverage edge-based and state-based intrinsic rewards in a joint way to improve exploration. Based on extensive experiments on the GridWorld task, we demonstrate the effectiveness and efficiency of GAFlowNet in terms of convergence, performance, and diversity of solutions. We further show that GAFlowNet is scalable to a more complex and large-scale molecule generation domain, where it achieves consistent and significant performance improvement.

2023-02-01

ICLR.cc/2023/Conference (notable)

Investigating Multi-task Pretraining and Generalization in Reinforcement Learning

Adrien Ali Taiga

Rishabh Agarwal

Jesse Farebrother

Google Brain

2023-02-01

ICLR.cc/2023/Conference (poster)

Latent State Marginalization as a Low-cost Approach for Improving Exploration

Dinghuai Zhang

Yoshua Bengio

Qinqing Zheng

Amy Zhang

Ricky T. Q. Chen

2023-02-01

ICLR.cc/2023/Conference (poster)

Online black-box adaptation to label-shift in the presence of conditional-shift

Faruk Ahmed

We consider an out-of-distribution setting where trained predictive models are deployed online in new locations (inducing conditional-shift)… (voir plus), such that these locations are also associated with differently skewed target distributions (label-shift). While approaches for online adaptation to label-shift have recently been discussed by Wu et al. (2021), the potential presence of concurrent conditional-shift has not been considered in the literature, although one might anticipate such distributional shifts in realistic deployments. In this paper, we empirically explore the effectiveness of online adaptation methods in such situations on three synthetic and two realistic datasets, comprising both classification and regression problems. We show that it is possible to improve performance in these settings by learning additional hyper-parameters to account for the presence of conditional-shift by using appropriate validation sets.

2023-02-01

ICLR.cc/2023/Conference (rejected)

Sample-Efficient Reinforcement Learning by Breaking the Replay Ratio Barrier

Pierluca D'Oro

Max Schwarzer

Evgenii Nikishin

Pierre-Luc Bacon

Increasing the replay ratio, the number of updates of an agent's parameters per environment interaction, is an appealing strategy for improv… (voir plus)ing the sample efficiency of deep reinforcement learning algorithms. In this work, we show that fully or partially resetting the parameters of deep reinforcement learning agents causes better replay ratio scaling capabilities to emerge. We push the limits of the sample efficiency of carefully-modified algorithms by training them using an order of magnitude more updates than usual, significantly improving their performance in the Atari 100k and DeepMind Control Suite benchmarks. We then provide an analysis of the design choices required for favorable replay ratio scaling to be possible and discuss inherent limits and tradeoffs.

2023-02-01

ICLR.cc/2023/Conference (notable)

Simplicial Embeddings in Self-Supervised Learning and Downstream Classification

Samuel Lavoie

Christos Tsirigotis

Max Schwarzer

Kenji Kawaguchi

Ankit Vani

Michael Noukhovitch

Simplicial Embeddings (SEM) are representations learned through self-supervised learning (SSL), wherein a representation is projected into …

2023-02-01

ICLR.cc/2023/Conference (notable)

Bigger, Better, Faster: Human-level Atari with human-level efficiency

Max Schwarzer

Johan Samir Obando Ceron

Rishabh Agarwal

Pablo Samuel Castro

We introduce a value-based RL agent, which we call BBF, that achieves super-human performance in the Atari 100K benchmark. BBF relies on sca… (voir plus)ling the neural networks used for value estimation, as well as a number of other design choices that enable this scaling in a sample-efficient manner. We conduct extensive analyses of these design choices and provide insights for future work. We end with a discussion about updating the goalposts for sample-efficient RL research on the ALE. We make our code and data publicly available at https://github.com/google-research/google-research/tree/master/bigger_better_faster.

2023-01-01

ICML (publié)

Noisy Pairing and Partial Supervision for Stylized Opinion Summarization

Reinald Kim

Mirella Lapata. 2020

Un-611

David Scott Krueger

Emmanuel Bengio

Maxinder S. Kan-620

Tegan Maharaj

Asja Fischer

Somnath Basu

Roy Chowdhury

Chao Zhao

Tanya Goyal

Junyi Jiacheng Xu

Jessy Li

Ivor Wai-hung Tsang

James T. Kwok

Neil Houlsby

Andrei Giurgiu

Stanisław Jastrzębski … (voir 22 de plus)

Bruna Morrone

Quentin de Laroussilhe

Mona Gesmundo

Attariyan Sylvain

Gelly

Thomas Wolf

Lysandre Debut

Julien Victor Sanh

Clement Chaumond

Anthony Delangue

Pier-339 Moi

Tim ric Cistac

R´emi Rault

Morgan Louf

Funtow-900 Joe

Sam Davison

Patrick Shleifer

Von Platen

Clara Ma

Yacine Jernite

Julien Plu

Canwen Xu

Opinion summarization research has primar-001 ily focused on generating summaries reflect-002 ing important opinions from customer reviews 0… (voir plus)03 without paying much attention to the writing 004 style. In this paper, we propose the stylized 005 opinion summarization task, which aims to 006 generate a summary of customer reviews in 007 the desired (e.g., professional) writing style. 008 To tackle the difficulty in collecting customer 009 and professional review pairs, we develop a 010 non-parallel training framework, Noisy Pair-011 ing and Partial Supervision ( NAPA ), which 012 trains a stylized opinion summarization sys-013 tem from non-parallel customer and profes-014 sional review sets. We create a benchmark P RO - 015 S UM by collecting customer and professional 016 reviews from Yelp and Michelin. Experimental 017 results on P RO S UM and FewSum demonstrate 018 that our non-parallel training framework con-019 sistently improves both automatic and human 020 evaluations, successfully building a stylized 021 opinion summarization model that can gener-022 ate professionally-written summaries from cus-023 tomer reviews. 024

2023-01-01

(publié)

www.semanticscholar.org

Versatile Energy-Based Models for High Energy Physics

Taoli Cheng

2023-01-01

arXiv.org (prépublication)