Publications

Multiset-Equivariant Set Prediction with Approximate Implicit Differentiation
Yan Zhang
David W. Zhang
Simon Lacoste-Julien
Gertjan J. Burghouts
Cees G. M. Snoek
Most set prediction models in deep learning use set-equivariant operations, but they actually operate on multisets. We show that set-equivar… (voir plus)iant functions cannot represent certain functions on multisets, so we introduce the more appropriate notion of multiset-equivariance. We identify that the existing Deep Set Prediction Network (DSPN) can be multiset-equivariant without being hindered by set-equivariance and improve it with approximate implicit differentiation, allowing for better optimization while being faster and saving memory. In a range of toy experiments, we show that the perspective of multiset-equivariance is beneficial and that our changes to DSPN achieve better results in most cases. On CLEVR object property prediction, we substantially improve over the state-of-the-art Slot Attention from 8% to 77% in one of the strictest evaluation metrics because of the benefits made possible by implicit differentiation.
2022-04-24
International Conference on Learning Representations (Accept (Poster))
doi.org
openreview.net
New Insights on Reducing Abrupt Representation Change in Online Continual Learning
Lucas Caccia
Rahaf Aljundi
Nader Asadi
Tinne Tuytelaars
Joelle Pineau
Eugene Belilovsky
In the online continual learning paradigm, agents must learn from a changing distribution while respecting memory and compute constraints. E… (voir plus)xperience Replay (ER), where a small subset of past data is stored and replayed alongside new data, has emerged as a simple and effective learning strategy. In this work, we focus on the change in representations of observed data that arises when previously unobserved classes appear in the incoming data stream, and new classes must be distinguished from previous ones. We shed new light on this question by showing that applying ER causes the newly added classes' representations to overlap significantly with the previous classes, leading to highly disruptive parameter updates. Based on this empirical analysis, we propose a new method which mitigates this issue by shielding the learned representations from drastic adaptation to accommodate new classes. We show that using an asymmetric update rule pushes new classes to adapt to the older ones (rather than the reverse), which is more effective especially at task boundaries, where much of the forgetting typically occurs. Empirical results show significant gains over strong baselines on standard continual learning benchmarks
2022-04-24
International Conference on Learning Representations (Accept (Poster))
doi.org
openreview.net
I NTRODUCING C OORDINATION IN C ONCURRENT R EIN - FORCEMENT L EARNING
Adrien Ali Taiga
Aaron Courville
Bellemare Marc-Emmanuel
Google Brain
Research on exploration in reinforcement learning has mostly focused on problems with a single agent interacting with an environment. Howeve… (voir plus)r many problems are better addressed by the concurrent reinforcement learning paradigm, where multiple agents operate in a common environment. Recent work has tackled the challenge of exploration in this particular setting (Dimakopoulou & Van Roy, 2018; Dimakopoulou et al., 2018). Nonetheless, they do not completely leverage the characteristics of this framework and agents end up behaving independently from each other. In this work we argue that coordination among concurrent agents is crucial for efficient exploration. We introduce coordination in Thompson Sampling based methods by drawing correlated samples from an agent’s posterior. We apply this idea to extend existing exploration schemes such as randomized least squares value iteration (RLSVI). Empirical results on simple toy tasks emphasize the merits of our approach and call attention to coordination as a key objective for efficient exploration in concurrent reinforcement learning.
2022-04-24
ICLR.cc/2022/Workshop/GMS (accepté)
openreview.net
openreview.net
Online Adversarial Attacks
Andjela Mladenovic
Avishek Joey Bose
Hugo Berard
William L. Hamilton
Simon Lacoste-Julien
Pascal Vincent
Gauthier Gidel
Adversarial attacks expose important vulnerabilities of deep learning models, yet little attention has been paid to settings where data arri… (voir plus)ves as a stream. In this paper, we formalize the online adversarial attack problem, emphasizing two key elements found in real-world use-cases: attackers must operate under partial knowledge of the target model, and the decisions made by the attacker are irrevocable since they operate on a transient data stream. We first rigorously analyze a deterministic variant of the online threat model by drawing parallels to the well-studied
2022-04-24
International Conference on Learning Representations (Accept (Poster))
doi.org
openreview.net
Policy Gradients Incorporating the Future
David Venuto
Elaine Lau
Doina Precup
Ofir Nachum
Reasoning about the future -- understanding how decisions in the present time affect outcomes in the future -- is one of the central challen… (voir plus)ges for reinforcement learning (RL), especially in highly-stochastic or partially observable environments. While predicting the future directly is hard, in this work we introduce a method that allows an agent to "look into the future" without explicitly predicting it. Namely, we propose to allow an agent, during its training on past experience, to observe what \emph{actually} happened in the future at that time, while enforcing an information bottleneck to avoid the agent overly relying on this privileged information. This gives our agent the opportunity to utilize rich and useful information about the future trajectory dynamics in addition to the present. Our method, Policy Gradients Incorporating the Future (PGIF), is easy to implement and versatile, being applicable to virtually any policy gradient algorithm. We apply our proposed method to a number of off-the-shelf RL algorithms and show that PGIF is able to achieve higher reward faster in a variety of online and offline RL domains, as well as sparse-reward and partially observable environments.
2022-04-24
International Conference on Learning Representations (Accept (Poster))
doi.org
openreview.net
Pre-training Molecular Graph Representation with 3D Geometry
Shengchao Liu
Hanchen Wang
Weiyang Liu
Joan Lasenby
Hongyu Guo
Jian Tang
Molecular graph representation learning is a fundamental problem in modern drug and material discovery. Molecular graphs are typically model… (voir plus)ed by their 2D topological structures, but it has been recently discovered that 3D geometric information plays a more vital role in predicting molecular functionalities. However, the lack of 3D information in real-world scenarios has significantly impeded the learning of geometric graph representation. To cope with this challenge, we propose the Graph Multi-View Pre-training (GraphMVP) framework where self-supervised learning (SSL) is performed by leveraging the correspondence and consistency between 2D topological structures and 3D geometric views. GraphMVP effectively learns a 2D molecular graph encoder that is enhanced by richer and more discriminative 3D geometry. We further provide theoretical insights to justify the effectiveness of GraphMVP. Finally, comprehensive experiments show that GraphMVP can consistently outperform existing graph SSL methods. Code is available on GitHub: https://github.com/chao1224/GraphMVP.
2022-04-24
International Conference on Learning Representations (Accept (Poster))
openreview.net
openreview.net
Properties from Mechanisms: An Equivariance Perspective on Identifiable Representation Learning
Kartik Ahuja
Jason Hartford
Yoshua Bengio
A key goal of unsupervised representation learning is "inverting" a data generating process to recover its latent properties. Existing work … (voir plus)that provably achieves this goal relies on strong assumptions on relationships between the latent variables (e.g., independence conditional on auxiliary information). In this paper, we take a very different perspective on the problem and ask, "Can we instead identify latent properties by leveraging knowledge of the mechanisms that govern their evolution?" We provide a complete characterization of the sources of non-identifiability as we vary knowledge about a set of possible mechanisms. In particular, we prove that if we know the exact mechanisms under which the latent properties evolve, then identification can be achieved up to any equivariances that are shared by the underlying mechanisms. We generalize this characterization to settings where we only know some hypothesis class over possible mechanisms, as well as settings where the mechanisms are stochastic. We demonstrate the power of this mechanism-based perspective by showing that we can leverage our results to generalize existing identifiable representation learning results. These results suggest that by exploiting inductive biases on mechanisms, it is possible to design a range of new identifiable representation learning approaches.
2022-04-24
International Conference on Learning Representations (Accept (Spotlight))
doi.org
openreview.net
QEN: Applicable Taxonomy Completion via Evaluating Full Taxonomic Relations
Suyuchen Wang
Ruihui Zhao
Yefeng Zheng
Bang Liu
Taxonomy is a fundamental type of knowledge graph for a wide range of web applications like searching and recommendation systems. To keep a … (voir plus)taxonomy automatically updated with the latest concepts, the taxonomy completion task matches a pair of proper hypernym and hyponym in the original taxonomy with the new concept as its parent and child. Previous solutions utilize term embeddings as input and only evaluate the parent-child relations between the new concept and the hypernym-hyponym pair. Such methods ignore the important sibling relations, and are not applicable in reality since term embeddings are not available for the latest concepts. They also suffer from the relational noise of the “pseudo-leaf” node, which is a null node acting as a node’s hyponym to enable the new concept to be a leaf node. To tackle the above drawbacks, we propose the Quadruple Evaluation Network (QEN), a novel taxonomy completion framework that utilizes easily accessible term descriptions as input, and applies pretrained language model and code attention for accurate inference while reducing online computation. QEN evaluates both parent-child and sibling relations to both enhance the accuracy and reduce the noise brought by pseudo-leaf. Extensive experiments on three real-world datasets in different domains with different sizes and term description sources prove the effectiveness and robustness of QEN on overall performance and especially the performance for adding non-leaf nodes, which largely surpasses previous methods and achieves the new state-of-the-art of the task.1
2022-04-24
The Web Conference (publié)
doi.org
Rare CNVs and phenome-wide profiling: a tale of brain-structural divergence and phenotypical convergence
J. Kopal
K. Kumar
K. Saltoun
C. Modenato
C. A. Moreau
S. Martin-Brevet
G. Huguet
M. Jean-Louis
C.O. Martin
Z. Saci
N. Younis
P. Tamer
E. Douard
A. M. Maillard
B. Rodriguez-Herreros
A. Pain
S. Richetin
L. Kushan
A. I. Silva
M. B. M. van den Bree … (voir 12 de plus)
D. E. J. Linden
M. J. Owen
J. Hall
S. Lippé
B. Draganski
I. E. Sønderby
O. A. Andreassen
D. C. Glahn
P. M. Thompson
C. E. Bearden
S. Jacquemont
D. Bzdok
Copy number variations (CNVs) are rare genomic deletions and duplications that can exert profound effects on brain and behavior. Previous re… (voir plus)ports of pleiotropy in CNVs imply that they converge on shared mechanisms at some level of pathway cascades, from genes to large-scale neural circuits to the phenome. However, studies to date have primarily examined single CNV loci in small clinical cohorts. It remains unknown how distinct CNVs escalate the risk for the same developmental and psychiatric disorders. Here, we quantitatively dissect the impact on brain organization and behavioral differentiation across eight key CNVs. In 534 clinical CNV carriers from multiple sites, we explored CNV-specific brain morphology patterns. We extensively annotated these CNV-associated patterns with deep phenotyping assays through the UK Biobank resource. Although the eight CNVs cause disparate brain changes, they are tied to similar phenotypic profiles across ∼1000 lifestyle indicators. Our population-level investigation established brain structural divergences and phenotypical convergences of CNVs, with direct relevance to major brain disorders.
2022-04-24
bioRxiv (prépublication)
doi.org
Sample Efficient Deep Reinforcement Learning via Uncertainty Estimation
Vincent Mai
Kaustubh Mani
Liam Paull
In model-free deep reinforcement learning (RL) algorithms, using noisy value estimates to supervise policy evaluation and optimization is de… (voir plus)trimental to the sample efficiency. As this noise is heteroscedastic, its effects can be mitigated using uncertainty-based weights in the optimization process. Previous methods rely on sampled ensembles, which do not capture all aspects of uncertainty. We provide a systematic analysis of the sources of uncertainty in the noisy supervision that occurs in RL, and introduce inverse-variance RL, a Bayesian framework which combines probabilistic ensembles and Batch Inverse Variance weighting. We propose a method whereby two complementary uncertainty estimation methods account for both the Q-value and the environment stochasticity to better mitigate the negative impacts of noisy supervision. Our results show significant improvement in terms of sample efficiency on discrete and continuous control tasks.
2022-04-24
International Conference on Learning Representations (Accept (Spotlight))
doi.org
openreview.net
Shared and unique brain network features predict cognitive, personality, and mental health scores in the ABCD study
Jianzhong Chen
Angela Tam
Valeria Kebets
Csaba Orbán
Leon Qi Rong Ooi
Christopher L. Asplund
Scott Marek
Nico U.F. Dosenbach
Simon B. Eickhoff
Danilo Bzdok
Avram J. Holmes
B.T. Thomas Yeo
How individual differences in brain network organization track behavioral variability is a fundamental question in systems neuroscience. Rec… (voir plus)ent work suggests that resting-state and task-state functional connectivity can predict specific traits at the individual level. However, most studies focus on single behavioral traits, thus not capturing broader relationships across behaviors. In a large sample of 1858 typically developing children from the Adolescent Brain Cognitive Development (ABCD) study, we show that predictive network features are distinct across the domains of cognitive performance, personality scores and mental health assessments. On the other hand, traits within each behavioral domain are predicted by similar network features. Predictive network features and models generalize to other behavioral measures within the same behavioral domain. Although tasks are known to modulate the functional connectome, predictive network features are similar between resting and task states. Overall, our findings reveal shared brain network features that account for individual variation within broad domains of behavior in childhood.
2022-04-24
Nature Communications (publié)
doi.org
Staged independent learning: Towards decentralized cooperative multi-agent Reinforcement Learning
Hadi Nekoei
Akilesh Badrinaaraayanan
Amit Sinha
Mohammad Amini
Janarthanan Rajendran
Aditya Mahajan
A. Chandar
We empirically show that classic ideas from two-time scale stochastic approximation \citep{borkar1997stochastic} can be combined with sequen… (voir plus)tial iterative best response (SIBR) to solve complex cooperative multi-agent reinforcement learning (MARL) problems. We first start with giving a multi-agent estimation problem as a motivating example where SIBR converges while parallel iterative best response (PIBR) does not. Then we present a general implementation of staged multi-agent RL algorithms based on SIBR and multi-time scale stochastic approximation, and show that our new methods which we call Staged Independent Proximal Policy Optimization (SIPPO) and Staged Independent Q-learning (SIQL) outperform state-of-the-art independent learning on almost all the tasks in the epymarl \citep{papoudakis2020benchmarking} benchmark. This can be seen as a first step towards more decentralized MARL methods based on SIBR and multi-time scale learning.
2022-04-24
ICLR.cc/2022/Workshop/GMS (accepté)
openreview.net
openreview.net