Publications

Systematic Generalization with Edge Transformers
Leon Bergen
Timothy J. O'Donnell
Dzmitry Bahdanau
Recent research suggests that systematic generalization in natural language understanding remains a challenge for state-of-the-art neural mo… (see more)dels such as Transformers and Graph Neural Networks. To tackle this challenge, we propose Edge Transformer, a new model that combines inspiration from Transformers and rule-based symbolic AI. The first key idea in Edge Transformers is to associate vector states with every edge, that is, with every pair of input nodes -- as opposed to just every node, as it is done in the Transformer model. The second major innovation is a triangular attention mechanism that updates edge representations in a way that is inspired by unification from logic programming. We evaluate Edge Transformer on compositional generalization benchmarks in relational reasoning, semantic parsing, and dependency parsing. In all three settings, the Edge Transformer outperforms Relation-aware, Universal and classical Transformer baselines.
2021-11-30
arXiv (Cornell University) (unknown)
doi.org
arxiv.org
Tau‐load in the lingual gyrus impacts anxiety levels during the COVID‐19 pandemic in participants of longitudinal observational studies in aging
Stijn Servaes
Firoza Z Lussier
Gleb Bezgin
Yi‐Ting Wang
Jenna Stevenson
Cécile Tissot
Guillaume Elgbeili
Jaime Fernandez Arias
Joseph Therriault
Andréa Lessa Benedet
Mira Chamoun
Tharick A. Pascoal
Danilo Bzdok
Suzanne King
Serge Gauthier
Pedro Rosa‐Neto
By obtaining a better grasp on the impact of the COVID‐19 pandemic on individuals with cognitive impairment, this knowledge could be used … (see more)to improve the delivery of information to this particular group. We aimed to assess the relationship between tau deposition and the change in anxiety levels, before and during the pandemic. We hypothesized that since the pandemic, higher tau loads would lower the change in anxiety. Furthermore, we expected these anxiety levels not to be associated with COVID‐19 related stress in participants with cognitive decline. 63 participants of the Translational Biomarker of Aging and Dementia (TRIAD) cohort (cognitively healthy, N=38; cognitively impaired, N=25, of which 7 had dementia due to Alzheimer’s disease), were assessed to evaluate their individual change in anxiety levels (GAD‐7). This was done at three different timepoints, of which the latest fell during the COVID‐19 lockdown period. Two rates of change, one before and one during the pandemic, were determined using the following definition: (next timepoint – current timepoint)/time difference. In addition, at the latest timepoint, subjective stress due to COVID‐19 was measured using the Montreal Assessment of Stress related to COVID‐19 (MASC). To assess the levels of tau, standard uptake value ratios (SUVR) from previously obtained [18F]MK‐6240 PET‐scans were used. [18F]MK‐6240 tracer binding in the lingual gyrus was negatively associated with the rate of change in GAD‐7 scores after correcting for age, sex, years of education and the presence of APOE ε4, but only in cognitively impaired individuals during the pandemic (fig 1A). In addition, the GAD‐7 score at the latest timepoint was associated with stress related to COVID‐19, but only in cognitively healthy individuals (fig 1B and 1C). The presence of tau in the lingual gyrus negatively affected the rate of change in GAD‐7 scores during the COVID‐19 pandemic in individuals with cognitive impairment. This could indicate that information pertaining to the pandemic does not reach these individuals in an efficient manner. The missing association between COVID‐19 induced stress and the latest GAD‐7 scores in these individuals is a further indication of this.
2021-11-30
Alzheimer's & Dementia (unknown)
doi.org
Hypo- and hyper- sensory processing heterogeneity in Autism Spectrum Disorder
Aline Lefebvre
Julian Tillmann
Freddy Cliquet
Frederique Amsellem
Anna Maruani
Claire Leblond
Anita Beggiato
David Germanaud
Anouck Amestoy
Myriam Ly‐Le Moal
Daniel Umbricht
Christopher Chattam
Lorraine Murtagh
Manuel Bouvard
Marion Leboyer
Tony Charman
Thomas Bourgeron
Richard Delorme
Guillaume Dumas
Background. Sensory processing atypicalities are part of the core symptoms of autism spectrum disorder (ASD) and could result from an excita… (see more)tion/inhibition imbalance. Yet, the convergence level of phenotypic sensory processing atypicalities with genetic alterations in GABA-ergic and glutamatergic pathways remains poorly understood. This study aimed to characterize the distribution of hypo/hyper-sensory profile among individuals with ASD and investigate the role of deleterious mutations in GABAergic and glutamatergic pathways related genes in sensory processing atypicalities. Method. From the Short Sensory Profile (SSP) questionnaire, we defined and explored a score – the differential Short Sensory Profile (dSSP) - as a normalized and centralized hypo/hypersensitivity ratio for 1136 participants (533 with ASD, 210 first-degree relatives, and 267 controls) from two independent study samples (PARIS and LEAP). We also performed an unsupervised item-based clustering analysis on SSP items scores to validate this new categorization in terms of hypo and hyper sensitivity. We then explored the link between the dSSP score and the burden of deleterious mutations in a subset of individuals for which whole-genome sequencing data were available. Results. We observed a mean dSSP score difference between ASD and controls, driven mostly by a high dSSP score variability among groups (PARIS: p0.0001, η2 = 0.0001, LEAP: p0.0001, Cohen’s d=3.67). First-degree relatives were with an intermediate distribution variability prof
2021-11-29
(published)
doi.org
Fixing Bias in Reconstruction-Based Anomaly Detection with Lipschitz Discriminators
Alexander Tong
Guy Wolf
Smita Krishnaswamy
Anomaly detection is of great interest in fields where abnormalities need to be identified and corrected (e.g., medicine and finance). Deep … (see more)learning methods for this task often rely on autoencoder reconstruction error, sometimes in conjunction with other errors. We show that this approach exhibits intrinsic biases that lead to undesirable results. Reconstruction-based methods are sensitive to training-data outliers and simple-to-reconstruct points. Instead, we introduce a new unsupervised Lipschitz anomaly discriminator that does not suffer from these biases. Our anomaly discriminator is trained, similar to the ones used in GANs, to detect the difference between the training data and corruptions of the training data. We show that this procedure successfully detects unseen anomalies with guarantees on those that have a certain Wasserstein distance from the data or corrupted training set. These additions allow us to show improved performance on MNIST, CIFAR10, and health record data.
2021-11-27
Journal of Signal Processing Systems (unknown)
doi.org
arxiv.org
Loneliness is linked to specific subregional alterations in hippocampus-default network co-variation
Chris Zajner
R. Nathan Spreng
Danilo Bzdok
Social interaction complexity makes humans unique. But in times of social deprivation this strength risks to expose important vulnerabilitie… (see more)s. Human social neuroscience studies have placed a premium on the default network (DN). In contrast, hippocampus (HC) subfields have been intensely studied in rodents and monkeys. To bridge these two literatures, we here quantified how DN subregions systematically co-vary with specific HC subfields in the context of subjective social isolation (i.e., loneliness). By co-decomposition using structural brain scans of ∼40,000 UK Biobank participants, loneliness was specially linked to midline subregions in the uncovered DN patterns. These association cortex signatures coincided with concomitant HC patterns implicating especially CA1 and molecular layer. These patterns also showed a strong affiliation with the fornix white-matter tract and the nucleus accumbens. In addition, separable signatures of structural HC-DN co-variation had distinct associations with the genetic predisposition for loneliness at the population level.
2021-11-23
Journal of Neurophysiology (published)
doi.org
Digital Ageism: Challenges and Opportunities in Artificial Intelligence for Older Adults
Charlene H. Chu
Rune Nyrup
Kathleen Leslie
Jiamin Shi
Andria Bianchi
Alexandra Lyn
Molly McNicholl
Shehroz Khan
Samira Rahimi
Amanda Grenier
Artificial intelligence (AI) and machine learning are changing our world through their impact on sectors including health care, education, e… (see more)mployment, finance, and law. AI systems are developed using data that reflect the implicit and explicit biases of society, and there are significant concerns about how the predictive models in AI systems amplify inequity, privilege, and power in society. The widespread applications of AI have led to mainstream discourse about how AI systems are perpetuating racism, sexism, and classism; yet, concerns about ageism have been largely absent in the AI bias literature. Given the globally aging population and proliferation of AI, there is a need to critically examine the presence of age-related bias in AI systems. This forum article discusses ageism in AI systems and introduces a conceptual model that outlines intersecting pathways of technology development that can produce and reinforce digital ageism in AI systems. We also describe the broader ethical and legal implications and considerations for future directions in digital ageism research to advance knowledge in the field and deepen our understanding of how ageism in AI is fostered by broader cycles of injustice.
2021-11-14
The Gerontologist (published)
doi.org
Splitting, Renaming, Removing: A Study of Common Cleaning Activities in Jupyter Notebooks
Helen Dong
Shurui Zhou
Jin L.C. Guo
Christian Kästner
Data scientists commonly use computational notebooks because they provide a good environment for testing multiple models. However, once the … (see more)scientist completes the code and finds the ideal model, he or she will have to dedicate time to clean up the code in order for others to easily understand it. In this paper, we perform a qualitative study on how scientists clean their code in hopes of being able to suggest a tool to automate this process. Our end goal is for tool builders to address possible gaps and provide additional aid to data scientists, who then can focus more on their actual work rather than the routine and tedious cleaning work. By sampling notebooks from GitHub and analyzing changes between subsequent commits, we identified common cleaning activities, such as changes to markdown (e.g., adding headers sections or descriptions) or comments (both deleting dead code and adding descriptions) as well as reordering cells. We also find that common cleaning activities differ depending on the intended purpose of the notebook. Our results provide a valuable foundation for tool builders and notebook users, as many identified cleaning activities could benefit from codification of best practices and dedicated tool support, possibly tailored depending on intended use.
2021-11-14
2021 36th IEEE/ACM International Conference on Automated Software Engineering Workshops (ASEW) (published)
doi.org
Subtle Bugs Everywhere: Generating Documentation for Data Wrangling Code
Chenyang Yang
Shurui Zhou
Jin L.C. Guo
Christian Kästner
Data scientists reportedly spend a significant amount of their time in their daily routines on data wrangling, i.e. cleaning data and extrac… (see more)ting features. However, data wrangling code is often repetitive and error-prone to write. Moreover, it is easy to introduce subtle bugs when reusing and adopting existing code, which results in reduced model quality. To support data scientists with data wrangling, we present a technique to generate documentation for data wrangling code. We use (1) program synthesis techniques to automatically summarize data transformations and (2) test case selection techniques to purposefully select representative examples from the data based on execution information collected with tailored dynamic program analysis. We demonstrate that a JupyterLab extension with our technique can provide on-demand documentation for many cells in popular notebooks and find in a user study that users with our plugin are faster and more effective at finding realistic bugs in data wrangling code.
2021-11-14
2021 36th IEEE/ACM International Conference on Automated Software Engineering (ASE) (published)
doi.org
ZERO: Playing Mathematical Programming Games
Gabriele Dragotto
S. Sankaranarayanan
Margarida Carvalho
Andrea Lodi
2021-11-14
ArXiv (preprint)
arxiv.org
arxiv.org
Hidden Hypergraphs, Error-Correcting Codes, and Critical Learning in Hopfield Networks
Christopher Hillar
Tenzin Chan
Rachel Taubman
David Rolnick
In 1943, McCulloch and Pitts introduced a discrete recurrent neural network as a model for computation in brains. The work inspired breakthr… (see more)oughs such as the first computer design and the theory of finite automata. We focus on learning in Hopfield networks, a special case with symmetric weights and fixed-point attractor dynamics. Specifically, we explore minimum energy flow (MEF) as a scalable convex objective for determining network parameters. We catalog various properties of MEF, such as biological plausibility, and then compare to classical approaches in the theory of learning. Trained Hopfield networks can perform unsupervised clustering and define novel error-correcting coding schemes. They also efficiently find hidden structures (cliques) in graph theory. We extend this known connection from graphs to hypergraphs and discover n-node networks with robust storage of 2Ω(n1−ϵ) memories for any ϵ>0. In the case of graphs, we also determine a critical ratio of training samples at which networks generalize completely.
2021-11-10
Entropy (published)
doi.org
The Cut and Play Algorithm: Computing Nash Equilibria via Outer Approximations
Margarida Carvalho
Gabriele Dragotto
Andrea Lodi
Sriram Sankaranarayanan
We introduce the Cut-and-Play, an efficient algorithm for computing equilibria in simultaneous non-cooperative games where players solve non… (see more)convex and possibly unbounded optimization problems. Our algorithm exploits an intrinsic relationship between the equilibria of the original nonconvex game and the ones of a convexified counterpart. In practice, Cut-and-Play formulates a series of convex approximations of the original game and refines them with techniques from integer programming, for instance, cutting planes and branching operations. We test our algorithm on two families of challenging nonconvex games involving discrete decisions and bilevel programs, and we empirically demonstrate that it efficiently computes equilibria and outperforms existing game-specific algorithms.
2021-11-09
ArXiv (preprint)
arxiv.org
arxiv.org
S$^3$: Sign-Sparse-Shift Reparametrization for Effective Training of Low-bit Shift Networks
Xinlin Li
Bang Liu
Yaoliang Yu
Wulong Liu
Chunjing Xu
Vahid Partovi Nia
2021-11-08
NeurIPS.cc/2021/Conference (poster)
openreview.net
openreview.net