Publications

Discovering modular solutions that generalize compositionally
Simon Schug
Seijin Kobayashi
Yassir Akram
Maciej Wolczyk
Alexandra Proca
Johannes Von Oswald
Razvan Pascanu
João Sacramento
Angelika Steger
Many complex tasks can be decomposed into simpler, independent parts. Discovering such underlying compositional structure has the potential … (see more)to enable compositional generalization. Despite progress, our most powerful systems struggle to compose flexibly. It therefore seems natural to make models more modular to help capture the compositional nature of many tasks. However, it is unclear under which circumstances modular systems can discover hidden compositional structure. To shed light on this question, we study a teacher-student setting with a modular teacher where we have full control over the composition of ground truth modules. This allows us to relate the problem of compositional generalization to that of identification of the underlying modules. In particular we study modularity in hypernetworks representing a general class of multiplicative interactions. We show theoretically that identification up to linear transformation purely from demonstrations is possible without having to learn an exponential number of module combinations. We further demonstrate empirically that under the theoretically identified conditions, meta-learning from finite data can discover modular policies that generalize compositionally in a number of complex environments.
2023-12-31
International Conference on Learning Representations (published)
doi.org
arxiv.org
Disentangling the Causes of Plasticity Loss in Neural Networks
Clare Lyle
Zeyu Zheng
Khimya Khetarpal
Hado van Hasselt
Razvan Pascanu
James Martens
Will Dabney
2023-12-31
CoLLAs (published)
doi.org
proceedings.mlr.press
Dissecting Deep RL with High Update Ratios: Combatting Value Divergence.
Marcel Hussing
Claas Voelcker
Igor Gilitschenski
Amir-massoud Farahmand
Eric R. Eaton
2023-12-31
RLJ (published)
arxiv.org
arxiv.org
Doob's Lagrangian: A Sample-Efficient Variational Approach to Transition Path Sampling
Yuanqi Du
Michael Plainer
Rob Brekelmans
Chenru Duan
Frank Noé
Carla P. Gomes
Alán Aspuru-Guzik
Kirill Neklyudov
Rare event sampling in dynamical systems is a fundamental problem arising in the natural sciences, which poses significant computational cha… (see more)llenges due to an exponentially large space of trajectories. For settings where the dynamical system of interest follows a Brownian motion with known drift, the question of conditioning the process to reach a given endpoint or desired rare event is definitively answered by Doob's h-transform. However, the naive estimation of this transform is infeasible, as it requires simulating sufficiently many forward trajectories to estimate rare event probabilities. In this work, we propose a variational formulation of Doob's h-transform as an optimization problem over trajectories between a given initial point and the desired ending point. To solve this optimization, we propose a simulation-free training objective with a model parameterization that imposes the desired boundary conditions by design. Our approach significantly reduces the search space over trajectories and avoids expensive trajectory simulation and inefficient importance sampling estimators which are required in existing methods. We demonstrate the ability of our method to find feasible transition paths on real-world molecular simulation and protein folding tasks.
2023-12-31
arXiv (preprint)
doi.org
openreview.net
Dynamic Neural Control Flow Execution: An Agent-Based Deep Equilibrium Approach for Binary Vulnerability Detection
Litao Li
Steven H. H. Ding
Andrew Walenstein
Philippe Charland
Benjamin C. M. Fung
2023-12-31
CIKM (published)
doi.org
arxiv.org
Dynamic Routing and Wavelength Assignment with Reinforcement Learning.
Peyman Kafaei
Quentin Cappart
Nicolas Chapados
Hamed Pouya
Louis-Martin Rousseau
With the rapid developments in communication systems, and considering their dynamic nature, all-optical networks are becoming increasingly c… (see more)omplex. This study proposes a novel method based on deep reinforcement learning for the routing and wavelength assignment problem in all-optical wavelength-decision-multiplexing networks. We consider dynamic incoming requests, in which their arrival and holding times are not known in advance. The objective is to devise a strategy that minimizes the number of rejected packages due to the lack of resources in the long term. We use graph neural networks to capture crucial latent information from the graph-structured input to develop the optimal strategy. The proposed deep reinforcement learning algorithm selects a route and a wavelength simultaneously for each incoming traffic connection as they arrive. The results demonstrate that the learned agent outperforms the methods used in practice and can be generalized on network topologies that did not participate in training.
2023-12-31
INFORMS Journal on Optimization (published)
doi.org
E(3)-Equivariant Mesh Neural Networks
Thuan Trang
Nhat Khang Ngo
Daniel Levy
Thieu N. Vo
Siamak Ravanbakhsh
Truong Son Hy
Triangular meshes are widely used to represent three-dimensional objects. As a result, many recent works have address the need for geometric… (see more) deep learning on 3D mesh. However, we observe that the complexities in many of these architectures does not translate to practical performance, and simple deep models for geometric graphs are competitive in practice. Motivated by this observation, we minimally extend the update equations of E(n)-Equivariant Graph Neural Networks (EGNNs) (Satorras et al., 2021) to incorporate mesh face information, and further improve it to account for long-range interactions through hierarchy. The resulting architecture, Equivariant Mesh Neural Network (EMNN), outperforms other, more complicated equivariant methods on mesh tasks, with a fast run-time and no expensive pre-processing. Our implementation is available at https://github.com/HySonLab/EquiMesh
2023-12-31
AISTATS (published)
doi.org
proceedings.mlr.press
ECBD: Evidence-Centered Benchmark Design for NLP
Yu Lu Liu
Su Lin Blodgett
Jackie Chi
Jackie CK Cheung
Kit Cheung
Q. Vera Liao
A.R. Olteanu
Ziang Xiao
Benchmarking is seen as critical to assessing progress in NLP. However, creating a benchmark involves many design decisions (e.g., which dat… (see more)asets to include, which metrics to use) that often rely on tacit, untested assumptions about what the benchmark is intended to measure or is actually measuring. There is currently no principled way of analyzing these decisions and how they impact the validity of the benchmark's measurements. To address this gap, we draw on evidence-centered design in educational assessments and propose Evidence-Centered Benchmark Design (ECBD), a framework which formalizes the benchmark design process into five modules. ECBD specifies the role each module plays in helping practitioners collect evidence about capabilities of interest. Specifically, each module requires benchmark designers to describe, justify, and support benchmark design choices -- e.g., clearly specifying the capabilities the benchmark aims to measure or how evidence about those capabilities is collected from model responses. To demonstrate the use of ECBD, we conduct case studies with three benchmarks: BoolQ, SuperGLUE, and HELM. Our analysis reveals common trends in benchmark design and documentation that could threaten the validity of benchmarks' measurements.
2023-12-31
ACL (1) (published)
doi.org
arxiv.org
Efficient Reinforcement Learning by Discovering Neural Pathways
Samin Yeasar Arnob
Riyasat Ohib
Sergey Plis
Amy Zhang
Alessandro Sordoni
Doina Precup
2023-12-31
Advances in Neural Information Processing Systems 37 (published)
doi.org
openreview.net
Empirical Analysis of Model Selection for Heterogenous Causal Effect Estimation
Divyat Mahajan
Ioannis Mitliagkas
Brady Neal
Vasilis Syrgkanis
We study the problem of model selection in causal inference, specifically for the case of conditional average treatment effect (CATE) estima… (see more)tion under binary treatments. Unlike model selection in machine learning, there is no perfect analogue of cross-validation as we do not observe the counterfactual potential outcome for any data point. Towards this, there have been a variety of proxy metrics proposed in the literature, that depend on auxiliary nuisance models estimated from the observed data (propensity score model, outcome regression model). However, the effectiveness of these metrics has only been studied on synthetic datasets as we can access the counterfactual data for them. We conduct an extensive empirical analysis to judge the performance of these metrics introduced in the literature, and novel ones introduced in this work, where we utilize the latest advances in generative modeling to incorporate multiple realistic datasets. Our analysis suggests novel model selection strategies based on careful hyperparameter tuning of CATE estimators and causal ensembling.
2023-12-31
ICLR (published)
doi.org
arxiv.org
Enhancing Click-through Rate Prediction in Recommendation Domain with Search Query Representation
Yuening Wang
Man Chen
Yaochen Hu
Wei Guo
Yingxue Zhang
Huifeng Guo
Yang Liu
Mark J. Coates
2023-12-31
CIKM (published)
doi.org
arxiv.org
Enhancing Security and Energy Efficiency of Cyber-Physical Systems using Deep Reinforcement Learning
Saeid Jamshidi
Ashkan Amirnia
Amin Nikanjam
Foutse Khomh
2023-12-31
Procedia Computer Science (published)
doi.org