Publications

Auxiliary Losses for Learning Generalizable Concept-based Models
Ivaxi Sheth
S Ebrahimi Kahou
2023-09-20
NeurIPS.cc/2023/Conference (poster)
doi.org
openreview.net
Decision-Aware Actor-Critic with Function Approximation and Theoretical Guarantees
Sharan Vaswani
Amirreza Kazemi
Reza Babanezhad Harikandeh
Nicolas Roux
Actor-critic (AC) methods are widely used in reinforcement learning (RL) and benefit from the flexibility of using any policy gradient metho… (see more)d as the actor and value-based method as the critic. The critic is usually trained by minimizing the TD error, an objective that is potentially decorrelated with the true goal of achieving a high reward with the actor. We address this mismatch by designing a joint objective for training the actor and critic in a decision-aware fashion. We use the proposed objective to design a generic, AC algorithm that can easily handle any function approximation. We explicitly characterize the conditions under which the resulting algorithm guarantees monotonic policy improvement, regardless of the choice of the policy and critic parameterization. Instantiating the generic algorithm results in an actor that involves maximizing a sequence of surrogate functions (similar to TRPO, PPO) and a critic that involves minimizing a closely connected objective. Using simple bandit examples, we provably establish the benefit of the proposed critic objective over the standard squared error. Finally, we empirically demonstrate the benefit of our decision-aware actor-critic framework on simple RL problems.
2023-09-20
NeurIPS.cc/2023/Conference (poster)
doi.org
openreview.net
A Definition of Continual Reinforcement Learning
David Abel
Andre Barreto
Benjamin Van Roy
Doina Precup
Hado van Hasselt
Satinder Singh
2023-09-20
NeurIPS.cc/2023/Conference (poster)
doi.org
openreview.net
A Diffusion-Model of Joint Interactive Navigation
Matthew Niedoba
Jonathan Wilder Lavington
Yunpeng Liu
Vasileios Lioutas
Justice Sefas
Xiaoxuan Liang
Dylan Green
Setareh Dabiri
Berend Zwartsenberg
Adam Ścibior
Frank N. Wood
Simulation of autonomous vehicle systems requires that simulated traffic participants exhibit diverse and realistic behaviors. The use of pr… (see more)erecorded real-world traffic scenarios in simulation ensures realism but the rarity of safety critical events makes large scale collection of driving scenarios expensive. In this paper, we present DJINN - a diffusion based method of generating traffic scenarios. Our approach jointly diffuses the trajectories of all agents, conditioned on a flexible set of state observations from the past, present, or future. On popular trajectory forecasting datasets, we report state of the art performance on joint trajectory metrics. In addition, we demonstrate how DJINN flexibly enables direct test-time sampling from a variety of valuable conditional distributions including goal-based sampling, behavior-class sampling, and scenario editing.
2023-09-20
NeurIPS.cc/2023/Conference (poster)
doi.org
openreview.net
On Dynamic Programming Decompositions of Static Risk Measures in Markov Decision Processes
Jia Lin Hau
Erick Delage
Mohammad Ghavamzadeh
Marek Petrik
2023-09-20
NeurIPS.cc/2023/Conference (poster)
openreview.net
openreview.net
HyenaDNA: Long-Range Genomic Sequence Modeling at Single Nucleotide Resolution
Eric Nguyen
Michael Poli
Marjan Faizi
Armin W Thomas
Callum Birch-Sykes
Michael Wornow
Aman Patel
Clayton M. Rabideau
Stefano Massaroli
Yoshua Bengio
Stefano Ermon
Stephen Baccus
Christopher Re
2023-09-20
NeurIPS.cc/2023/Conference (spotlight)
doi.org
openreview.net
Importance-aware Co-teaching for Offline Model-based Optimization
Ye Yuan
Can Chen
Zixuan Liu
Willie Neiswanger
Xue Liu
2023-09-20
NeurIPS.cc/2023/Conference (poster)
openreview.net
openreview.net
Improving Language Plasticity via Pretraining with Active Forgetting
Yihong Chen
Kelly Marchisio
Roberta Raileanu
David Ifeoluwa Adelani
Pontus Stenetorp
Sebastian Riedel
Mikel Artetxe
Pretrained language models (PLMs) are today the primary model for natural language processing. Despite their impressive downstream performan… (see more)ce, it can be difficult to apply PLMs to new languages, a barrier to making their capabilities universally accessible. While prior work has shown it possible to address this issue by learning a new embedding layer for the new language, doing so is both data and compute inefficient. We propose to use an active forgetting mechanism during pretraining, as a simple way of creating PLMs that can quickly adapt to new languages. Concretely, by resetting the embedding layer every K updates during pretraining, we encourage the PLM to improve its ability of learning new embeddings within limited number of updates, similar to a meta-learning effect. Experiments with RoBERTa show that models pretrained with our forgetting mechanism not only demonstrate faster convergence during language adaptation, but also outperform standard ones in a low-data regime, particularly for languages that are distant from English. Code will be available at https://github.com/facebookresearch/language-model-plasticity.
2023-09-20
NeurIPS.cc/2023/Conference (poster)
doi.org
openreview.net
Laughing Hyena Distillery: Extracting Compact Recurrences From Convolutions
Stefano Massaroli
Michael Poli
Daniel Y Fu
Hermann Kumbong
Rom Nishijima Parnichkun
Aman Timalsina
David W. Romero
Quinn McIntyre
Beidi Chen
Atri Rudra
Ce Zhang
Christopher Re
Stefano Ermon
Yoshua Bengio
Recent advances in attention-free sequence models rely on convolutions as alternatives to the attention operator at the core of Transformers… (see more). In particular, long convolution sequence models have achieved state-of-the-art performance in many domains, but incur a significant cost during auto-regressive inference workloads -- naively requiring a full pass (or caching of activations) over the input sequence for each generated token -- similarly to attention-based models. In this paper, we seek to enable
2023-09-20
NeurIPS.cc/2023/Conference (poster)
doi.org
openreview.net
Learning Reliable Logical Rules with SATNet
Zhaoyu Li
Jinpei Guo
Yuhe Jiang
Xujie Si
2023-09-20
NeurIPS.cc/2023/Conference (poster)
doi.org
openreview.net
Multi-Head Adapter Routing for Cross-Task Generalization
Lucas Caccia
Edoardo Ponti
Zhan Su
Matheus Pereira
Nicolas Le Roux
Alessandro Sordoni
Parameter-efficient fine-tuning (PEFT) for cross-task generalization consists in pre-training adapters on a multi-task training set before f… (see more)ew-shot adaptation to test tasks. Polytropon [Ponti et al., 2023] (
2023-09-20
NeurIPS.cc/2023/Conference (poster)
doi.org
openreview.net
Neural Graph Generation from Graph Statistics.
Kiarash Zahirnia
Yaochen Hu
Mark J. Coates
Oliver Schulte
2023-09-20
NeurIPS.cc/2023/Conference (poster)
openreview.net
openreview.net