Publications

On the Systematicity of Probing Contextualized Word Representations: The Case of Hypernymy in BERT.
Abhilasha Ravichander
Eduard Hovy
Kaheer Suleman
Adam Trischler
Jackie CK Cheung
2020-11-30
STARSEM (published)
dblp.uni-trier.de
Probabilistic Classifiers
Irina Rish
2020-11-29
Data-Driven Computational Neuroscience (published)
doi.org
Universal Semantics for the Stochastic Lambda-Calculus
Pedro H. Azevedo de Amorim
Dexter Kozen
Radu Mardare
Prakash Panangaden
Michael Roberts
We define sound and adequate denotational and operational semantics for the stochastic lambda calculus. These two semantic approaches build … (see more)on previous work that used similar techniques to reason about higher-order probabilistic programs, but for the first time admit an adequacy theorem relating the operational and denotational views. This resolves the main issue left open in (Bacci et al. 2018).
2020-11-28
arXiv.org (preprint)
dblp.uni-trier.de
Human attachments shape interbrain synchrony toward efficient performance of social goals
Amir Djalovski
Guillaume Dumas
Sivan Kinreich
Ruth Pinkenson Feldman
2020-11-25
NeuroImage (published)
doi.org
Interactive Psychometrics for Autism With the Human Dynamic Clamp: Interpersonal Synchrony From Sensorimotor to Sociocognitive Domains
Florence Baillin
Aline Lefebvre
Amandine Pedoux
Yann Beauxis
Denis-Alexander Engemann
Anna Maruani
Frederique Amsellem
J. A. Scott Kelso
Thomas Bourgeron
Richard Delorme
Guillaume Dumas
The human dynamic clamp (HDC) is a human–machine interface designed on the basis of coordination dynamics for studying realistic social in… (see more)teraction under controlled and reproducible conditions. Here, we propose to probe the validity of the HDC as a psychometric instrument for quantifying social abilities in children with autism spectrum disorder (ASD) and neurotypical development. To study interpersonal synchrony with the HDC, we derived five standardized scores following a gradient from sensorimotor and motor to higher sociocognitive skills in a sample of 155 individuals (113 participants with ASD, 42 typically developing participants; aged 5 to 25 years; IQ > 70). Regression analyses were performed using normative modeling on global scores according to four subconditions (HDC behavior “cooperative/competitive,” human task “in-phase/anti-phase,” diagnosis, and age at inclusion). Children with ASD had lower scores than controls for motor skills. HDC motor coordination scores were the best candidates for stratification and diagnostic biomarkers according to exploratory analyses of hierarchical clustering and multivariate classification. Independently of phenotype, sociocognitive skills increased with developmental age while being affected by the ongoing task and HDC behavior. Weaker performance in ASD for motor skills suggests the convergent validity of the HDC for evaluating social interaction. Results provided additional evidence of a relationship between sensorimotor and sociocognitive skills. HDC may also be used as a marker of maturation of sociocognitive skills during real-time social interaction. Through its standardized and objective evaluation, the HDC not only represents a valid paradigm for the study of interpersonal synchrony but also offers a promising, clinically relevant psychometric instrument for the evaluation and stratification of sociomotor dysfunctions.
2020-11-25
Frontiers in Psychiatry (published)
doi.org
RetroGNN: Approximating Retrosynthesis by Graph Neural Networks for De Novo Drug Design
Cheng-Hao Liu
Maksym Korablyov
Stanisław Jastrzębski
Paweł Włodarczyk-Pruszyński
Yoshua Bengio
Marwin H. S. Segler
De novo molecule generation often results in chemically unfeasible molecules. A natural idea to mitigate this problem is to bias the search … (see more)process towards more easily synthesizable molecules using a proxy for synthetic accessibility. However, using currently available proxies still results in highly unrealistic compounds. We investigate the feasibility of training deep graph neural networks to approximate the outputs of a retrosynthesis planning software, and their use to bias the search process. We evaluate our method on a benchmark involving searching for drug-like molecules with antibiotic properties. Compared to enumerating over five million existing molecules from the ZINC database, our approach finds molecules predicted to be more likely to be antibiotics while maintaining good drug-like properties and being easily synthesizable. Importantly, our deep neural network can successfully filter out hard to synthesize molecules while achieving a
2020-11-24
arXiv (preprint)
doi.org
arxiv.org
Task dependent deep LDA pruning of neural networks
Qing Tian
Tal Arbel
James J. Clark
2020-11-23
Computer Vision and Image Understanding (published)
doi.org
AR-DAE: Towards Unbiased Neural Entropy Gradient Estimation
Jae Hyun Lim
Aaron Courville
Christopher Pal
Chin-wei Huang
Entropy is ubiquitous in machine learning, but it is in general intractable to compute the entropy of the distribution of an arbitrary conti… (see more)nuous random variable. In this paper, we propose the amortized residual denoising autoencoder (AR-DAE) to approximate the gradient of the log density function, which can be used to estimate the gradient of entropy. Amortization allows us to significantly reduce the error of the gradient approximator by approaching asymptotic optimality of a regular DAE, in which case the estimation is in theory unbiased. We conduct theoretical and experimental analyses on the approximation error of the proposed method, as well as extensive studies on heuristics to ensure its robustness. Finally, using the proposed gradient approximator to estimate the gradient of entropy, we demonstrate state-of-the-art performance on density estimation with variational autoencoders and continuous control with soft actor-critic.
2020-11-20
Proceedings of the 37th International Conference on Machine Learning (published)
doi.org
proceedings.mlr.press
Countering Language Drift with Seeded Iterated Learning
Yuchen Lu
Soumye Singhal
Florian Strub
Olivier Pietquin
Aaron Courville
Pretraining on human corpus and then finetuning in a simulator has become a standard pipeline for training a goal-oriented dialogue agent. N… (see more)evertheless, as soon as the agents are finetuned to maximize task completion, they suffer from the so-called language drift phenomenon: they slowly lose syntactic and semantic properties of language as they only focus on solving the task. In this paper, we propose a generic approach to counter language drift called Seeded iterated learning (SIL). We periodically refine a pretrained student agent by imitating data sampled from a newly generated teacher agent. At each time step, the teacher is created by copying the student agent, before being finetuned to maximize task completion. SIL does not require external syntactic constraint nor semantic knowledge, making it a valuable task-agnostic finetuning protocol. We evaluate SIL in a toy-setting Lewis Game, and then scale it up to the translation game with natural language. In both settings, SIL helps counter language drift as well as it improves the task completion compared to baselines.
2020-11-20
Proceedings of the 37th International Conference on Machine Learning (published)
doi.org
proceedings.mlr.press
Interference and Generalization in Temporal Difference Learning
Emmanuel Bengio
Joelle Pineau
Doina Precup
We study the link between generalization and interference in temporal-difference (TD) learning. Interference is defined as the inner product… (see more) of two different gradients, representing their alignment. This quantity emerges as being of interest from a variety of observations about neural networks, parameter sharing and the dynamics of learning. We find that TD easily leads to low-interference, under-generalizing parameters, while the effect seems reversed in supervised learning. We hypothesize that the cause can be traced back to the interplay between the dynamics of interference and bootstrapping. This is supported empirically by several observations: the negative relationship between the generalization gap and interference in TD, the negative effect of bootstrapping on interference and the local coherence of targets, and the contrast between the propagation rate of information in TD(0) versus TD(
2020-11-20
Proceedings of the 37th International Conference on Machine Learning (published)
doi.org
proceedings.mlr.press
Invariant Causal Prediction for Block MDPS
Amy Zhang
Clare Lyle
Shagun Sodhani
Angelos Filos
Marta Kwiatkowska
Joelle Pineau
Yarin Gal
Doina Precup
Generalization across environments is critical to the successful application of reinforcement learning algorithms to real-world challenges. … (see more)In this paper, we consider the problem of learning abstractions that generalize in block MDPs, families of environments with a shared latent state space and dynamics structure over that latent space, but varying observations. We leverage tools from causal inference to propose a method of invariant prediction to learn model-irrelevance state abstractions (MISA) that generalize to novel observations in the multi-environment setting. We prove that for certain classes of environments, this approach outputs with high probability a state abstraction corresponding to the causal feature set with respect to the return. We further provide more general bounds on model error and generalization error in the multi-environment setting, in the process showing a connection between causal variable selection and the state abstraction framework for MDPs. We give empirical evidence that our methods work in both linear and nonlinear settings, attaining improved generalization over single- and multi-task baselines.
2020-11-20
Proceedings of the 37th International Conference on Machine Learning (published)
doi.org
proceedings.mlr.press
Latent Variable Modelling with Hyperbolic Normalizing Flows
Avishek Joey Bose
Ariella Smofsky
Renjie Liao
Prakash Panangaden
William L. Hamilton
The choice of approximate posterior distributions plays a central role in stochastic variational inference (SVI). One effective solution is … (see more)the use of normalizing flows \cut{defined on Euclidean spaces} to construct flexible posterior distributions. However, one key limitation of existing normalizing flows is that they are restricted to the Euclidean space and are ill-equipped to model data with an underlying hierarchical structure. To address this fundamental limitation, we present the first extension of normalizing flows to hyperbolic spaces. We first elevate normalizing flows to hyperbolic spaces using coupling transforms defined on the tangent bundle, termed Tangent Coupling (
2020-11-20
Proceedings of the 37th International Conference on Machine Learning (published)
doi.org
proceedings.mlr.press