Publications

Decoupling regularization from the action space
Sobhan Mohammadpour
Emma Frejinger
Pierre-Luc Bacon
Regularized reinforcement learning (RL), particularly the entropy-regularized kind, has gained traction in optimal control and inverse RL. W… (see more)hile standard unregularized RL methods remain unaffected by changes in the number of actions, we show that it can severely impact their regularized counterparts. This paper demonstrates the importance of decoupling the regularizer from the action space: that is, to maintain a consistent level of regularization regardless of how many actions are involved to avoid over-regularization. Whereas the problem can be avoided by introducing a task-specific temperature parameter, it is often undesirable and cannot solve the problem when action spaces are state-dependent. In the state-dependent action context, different states with varying action spaces are regularized inconsistently. We introduce two solutions: a static temperature selection approach and a dynamic counterpart, universally applicable where this problem arises. Implementing these changes improves performance on the DeepMind control suite in static and dynamic temperature regimes and a biological design task.
2024-01-16
ICLR.cc/2024/Conference (poster)
openreview.net
openreview.net
Delta-AI: Local objectives for amortized inference in sparse graphical models
Jean-Pierre R. Falet
Hae Beom Lee
Nikolay Malkin
Chen Sun
Dragos Secrieru
Dinghuai Zhang
Guillaume Lajoie
Yoshua Bengio
We present a new algorithm for amortized inference in sparse probabilistic graphical models (PGMs), which we call …
2024-01-16
ICLR.cc/2024/Conference (poster)
doi.org
openreview.net
Diffusion Generative Flow Samplers: Improving learning signals through partial trajectory optimization
Dinghuai Zhang
Ricky T. Q. Chen
Cheng-Hao Liu
Aaron Courville
Yoshua Bengio
2024-01-16
ICLR.cc/2024/Conference (poster)
doi.org
openreview.net
On Diffusion Modeling for Anomaly Detection
Victor Livernoche
Vineet Jain
Yashar Hezaveh
Siamak Ravanbakhsh
Known for their impressive performance in generative modeling, diffusion models are attractive candidates for density-based anomaly detectio… (see more)n. This paper investigates different variations of diffusion modeling for unsupervised and semi-supervised anomaly detection. In particular, we find that Denoising Diffusion Probability Models (DDPM) are performant on anomaly detection benchmarks yet computationally expensive. By simplifying DDPM in application to anomaly detection, we are naturally led to an alternative approach called Diffusion Time Estimation (DTE). DTE estimates the distribution over diffusion time for a given input and uses the mode or mean of this distribution as the anomaly score. We derive an analytical form for this density and leverage a deep neural network to improve inference efficiency. Through empirical evaluations on the ADBench benchmark, we demonstrate that all diffusion-based anomaly detection methods perform competitively for both semi-supervised and unsupervised settings. Notably, DTE achieves orders of magnitude faster inference time than DDPM, while outperforming it on this benchmark. These results establish diffusion-based anomaly detection as a scalable alternative to traditional methods and recent deep-learning techniques for standard unsupervised and semi-supervised anomaly detection settings.
2024-01-16
ICLR.cc/2024/Conference (spotlight)
doi.org
openreview.net
Efficient Dynamics Modeling in Interactive Environments with Koopman Theory
Arnab Kumar Mondal
Siba Smarak Panigrahi
Sai Rajeswar
Kaleem Siddiqi
Siamak Ravanbakhsh
The accurate modeling of dynamics in interactive environments is critical for successful long-range prediction. Such a capability could adva… (see more)nce Reinforcement Learning (RL) and Planning algorithms, but achieving it is challenging. Inaccuracies in model estimates can compound, resulting in increased errors over long horizons. We approach this problem from the lens of Koopman theory, where the nonlinear dynamics of the environment can be linearized in a high-dimensional latent space. This allows us to efficiently parallelize the sequential problem of long-range prediction using convolution while accounting for the agent’s action at every time step. Our approach also enables stability analysis and better control over gradients through time. Taken together, these advantages result in significant improvement over the existing approaches, both in the efficiency and the accuracy of modeling dynamics over extended horizons. We also show that this model can be easily incorporated into dynamics modeling for model-based planning and model-free RL and report promising experimental results.
2024-01-16
ICLR.cc/2024/Conference (poster)
doi.org
openreview.net
Empirical Analysis of Model Selection for Heterogeneous Causal Effect Estimation
Divyat Mahajan
Ioannis Mitliagkas
Brady Neal
Vasilis Syrgkanis
2024-01-16
ICLR.cc/2024/Conference (spotlight)
openreview.net
openreview.net
Ensemble Distillation for Unsupervised Constituency Parsing
Behzad Shayegh
Yanshuai Cao
Xiaodan Zhu
Jackie Cheung
Lili Mou
2024-01-16
ICLR.cc/2024/Conference (poster)
openreview.net
openreview.net
Evaluating Representation Learning on the Protein Structure Universe
Arian Rokkum Jamasb
Alex Morehead
Chaitanya K. Joshi
Zuobai Zhang
Kieran Didi
Simon V Mathis
Charles Harris
Jian Tang
Jianlin Cheng
Pietro Lio
Tom Leon Blundell
2024-01-16
ICLR.cc/2024/Conference (poster)
openreview.net
openreview.net
Expected flow networks in stochastic environments and two-player zero-sum games
Marco Jiralerspong
Bilun Sun
Danilo Vucetic
Tianyu Zhang
Yoshua Bengio
Gauthier Gidel
Nikolay Malkin
2024-01-16
ICLR.cc/2024/Conference (poster)
doi.org
openreview.net
Ghost on the Shell: An Expressive Representation of General 3D Shapes
Zhen Liu
Yao Feng
Yuliang Xiu
Weiyang Liu
Liam Paull
Michael J. Black
Bernhard Schölkopf
2024-01-16
ICLR.cc/2024/Conference (oral)
doi.org
openreview.net
Hallucination Detection and Hallucination Mitigation: An Investigation
Junliang Luo
Tianyu Li
Di Wu
M. Jenkin
Steve Liu
Gregory Dudek
2024-01-16
ArXiv (preprint)
doi.org
arxiv.org
How connectivity structure shapes rich and lazy learning in neural circuits
Yuhan Helena Liu
Aristide Baratin
Jonathan Cornford
Stefan Mihalas
Eric Todd SheaBrown
Guillaume Lajoie
In theoretical neuroscience, recent work leverages deep learning tools to explore how some network attributes critically influence its learn… (see more)ing dynamics. Notably, initial weight distributions with small (resp. large) variance may yield a rich (resp. lazy) regime, where significant (resp. minor) changes to network states and representation are observed over the course of learning. However, in biology, neural circuit connectivity generally has a low-rank structure and therefore differs markedly from the random initializations generally used for these studies. As such, here we investigate how the structure of the initial weights — in particular their effective rank — influences the network learning regime. Through both empirical and theoretical analyses, we discover that high-rank initializations typically yield smaller network changes indicative of lazier learning, a finding we also confirm with experimentally-driven initial connectivity in recurrent neural networks. Conversely, low-rank initialization biases learning towards richer learning. Importantly, however, as an exception to this rule, we find lazier learning can still occur with a low-rank initialization that aligns with task and data statistics. Our research highlights the pivotal role of initial weight structures in shaping learning regimes, with implications for metabolic costs of plasticity and risks of catastrophic forgetting.
2024-01-16
ICLR.cc/2024/Conference (poster)
doi.org
openreview.net