Publications

A*Net: A Scalable Path-based Reasoning Approach for Knowledge Graphs

Zhaocheng Zhu

Xinyu Yuan

Mikhail Galkin

Louis-Pascal Xhonneux

Sophie Xhonneux

Ming Zhang

Maxime Gazeau

Jian Tang

openreview.net

Are Diffusion Models Vision-And-Language Reasoners?

Benno Krojer

Elinor Poole-Dayan

Vikram Voleti

Chris Pal

Siva Reddy

Text-conditioned image generation models have recently shown immense qualitative success using denoising diffusion processes. However, unlik… (see more)e discriminative vision-and-language models, it is a non-trivial task to subject these diffusion-based generative models to automatic fine-grained quantitative evaluation of high-level phenomena such as compositionality. Towards this goal, we perform two innovations. First, we transform diffusion-based models (in our case, Stable Diffusion) for any image-text matching (ITM) task using a novel method called DiffusionITM. Second, we introduce the Generative-Discriminative Evaluation Benchmark (GDBench) benchmark with 7 complex vision-and-language tasks, bias evaluation and detailed analysis. We find that Stable Diffusion + DiffusionITM is competitive on many tasks and outperforms CLIP on compositional tasks like like CLEVR and Winoground. We further boost its compositional performance with a transfer setup by fine-tuning on MS-COCO while retaining generative capabilities. We also measure the stereotypical bias in diffusion models, and find that Stable Diffusion 2.1 is, for the most part, less biased than Stable Diffusion 1.5. Overall, our results point in an exciting direction bringing discriminative and generative model evaluation closer. We will release code and benchmark setup soon.

openreview.net

Block-State Transformers

Jonathan Pilault

Mahan Fathi

Orhan Firat

Chris Pal

Pierre-Luc Bacon

Ross Goroshin

2023-09-21

NeurIPS.cc/2023/Conference (poster)

openreview.net

CADet: Fully Self-Supervised Out-Of-Distribution Detection With Contrastive Learning

Charles Guille-Escuret

Pau Rodriguez

David Vazquez

Ioannis Mitliagkas

Joao Monteiro

openreview.net

Contrastive Retrospection: honing in on critical steps for rapid learning and generalization in RL

Chen Sun

Wannan Yang

Thomas Jiralerspong

Dane Malenfant

Benjamin Alsbury-Nealy

Yoshua Bengio

Blake Richards

In real life, success is often contingent upon multiple critical steps that are distant in time from each other and from the final reward. T… (see more)hese critical steps are challenging to identify with traditional reinforcement learning (RL) methods that rely on the Bellman equation for credit assignment. Here, we present a new RL algorithm that uses offline contrastive learning to hone in on these critical steps. This algorithm, which we call Contrastive Retrospection (ConSpec), can be added to any existing RL algorithm. ConSpec learns a set of prototypes for the critical steps in a task by a novel contrastive loss and delivers an intrinsic reward when the current state matches one of the prototypes. The prototypes in ConSpec provide two key benefits for credit assignment: (i) They enable rapid identification of all the critical steps. (ii) They do so in a readily interpretable manner, enabling out-of-distribution generalization when sensory features are altered. Distinct from other contemporary RL approaches to credit assignment, ConSpec takes advantage of the fact that it is easier to retrospectively identify the small set of steps that success is contingent upon (and ignoring other states) than it is to prospectively predict reward at every taken step. ConSpec greatly improves learning in a diverse set of RL tasks. The code is available at the link: https://github.com/sunchipsster1/ConSpec

openreview.net

Decision-Aware Actor-Critic with Function Approximation and Theoretical Guarantees

Sharan Vaswani

Amirreza Kazemi

Reza Babanezhad Harikandeh

Nicolas Le Roux

Actor-critic (AC) methods are widely used in reinforcement learning (RL) and benefit from the flexibility of using any policy gradient metho… (see more)d as the actor and value-based method as the critic. The critic is usually trained by minimizing the TD error, an objective that is potentially decorrelated with the true goal of achieving a high reward with the actor. We address this mismatch by designing a joint objective for training the actor and critic in a decision-aware fashion. We use the proposed objective to design a generic, AC algorithm that can easily handle any function approximation. We explicitly characterize the conditions under which the resulting algorithm guarantees monotonic policy improvement, regardless of the choice of the policy and critic parameterization. Instantiating the generic algorithm results in an actor that involves maximizing a sequence of surrogate functions (similar to TRPO, PPO) and a critic that involves minimizing a closely connected objective. Using simple bandit examples, we provably establish the benefit of the proposed critic objective over the standard squared error. Finally, we empirically demonstrate the benefit of our decision-aware actor-critic framework on simple RL problems.

openreview.net

A Definition of Continual Reinforcement Learning

David Abel

Andre Barreto

Benjamin Van Roy

Doina Precup

Hado van Hasselt

Satinder Singh

openreview.net

DiffPack: A Torsional Diffusion Model for Autoregressive Protein Side-Chain Packing

Yang Zhang

Zuobai Zhang

Bozitao Zhong

Sanchit Misra

Jian Tang

Proteins play a critical role in carrying out biological functions, and their 3D structures are essential in determining their functions. A… (see more)ccurately predicting the conformation of protein side-chains given their backbones is important for applications in protein structure prediction, design and protein-protein interactions. Traditional methods are computationally intensive and have limited accuracy, while existing machine learning methods treat the problem as a regression task and overlook the restrictions imposed by the constant covalent bond lengths and angles. In this work, we present DiffPack, a torsional diffusion model that learns the joint distribution of side-chain torsional angles, the only degrees of freedom in side-chain packing, by diffusing and denoising on the torsional space. To avoid issues arising from simultaneous perturbation of all four torsional angles, we propose autoregressively generating the four torsional angles from

openreview.net

A Diffusion-Model of Joint Interactive Navigation

Matthew Niedoba

Jonathan Wilder Lavington

Yunpeng Liu

Vasileios Lioutas

Justice Sefas

Xiaoxuan Liang

Dylan Green

Setareh Dabiri

Berend Zwartsenberg

Adam Ścibior

Frank Wood

Simulation of autonomous vehicle systems requires that simulated traffic participants exhibit diverse and realistic behaviors. The use of pr… (see more)erecorded real-world traffic scenarios in simulation ensures realism but the rarity of safety critical events makes large scale collection of driving scenarios expensive. In this paper, we present DJINN - a diffusion based method of generating traffic scenarios. Our approach jointly diffuses the trajectories of all agents, conditioned on a flexible set of state observations from the past, present, or future. On popular trajectory forecasting datasets, we report state of the art performance on joint trajectory metrics. In addition, we demonstrate how DJINN flexibly enables direct test-time sampling from a variety of valuable conditional distributions including goal-based sampling, behavior-class sampling, and scenario editing.

openreview.net

Double Gumbel Q-Learning.

David Yu-Tung Hui

Aaron Courville

Pierre-Luc Bacon

openreview.net

On Dynamic Programming Decompositions of Static Risk Measures in Markov Decision Processes

Jia Lin Hau

Erick Delage

Mohammad Ghavamzadeh

Marek Petrik

openreview.net