Publications

Learning Efficient Task-Specific Meta-Embeddings with Word Prisms
Jingyi He
KC Tsiolis
Kian Kenyon-Dean
Jackie Chi Kit Cheung
Word embeddings are trained to predict word cooccurrence statistics, which leads them to possess different lexical properties (syntactic, se… (voir plus)mantic, etc.) depending on the notion of context defined at training time. These properties manifest when querying the embedding space for the most similar vectors, and when used at the input layer of deep neural networks trained to solve downstream NLP problems. Meta-embeddings combine multiple sets of differently trained word embeddings, and have been shown to successfully improve intrinsic and extrinsic performance over equivalent models which use just one set of source embeddings. We introduce word prisms: a simple and efficient meta-embedding method that learns to combine source embeddings according to the task at hand. Word prisms learn orthogonal transformations to linearly combine the input source embeddings, which allows them to be very efficient at inference time. We evaluate word prisms in comparison to other meta-embedding methods on six extrinsic evaluations and observe that word prisms offer improvements in performance on all tasks.
2020-11-30
Proceedings of the 28th International Conference on Computational Linguistics (publié)
doi.org
arxiv.org
Learning Lexical Subspaces in a Distributional Vector Space
Kushal Arora
Aishik Chakraborty
Jackie C. K. Cheung
In this paper, we propose LexSub, a novel approach towards unifying lexical and distributional semantics. We inject knowledge about lexical-… (voir plus)semantic relations into distributional word embeddings by defining subspaces of the distributional vector space in which a lexical relation should hold. Our framework can handle symmetric attract and repel relations (e.g., synonymy and antonymy, respectively), as well as asymmetric relations (e.g., hypernymy and meronomy). In a suite of intrinsic benchmarks, we show that our model outperforms previous approaches on relatedness tasks and on hypernymy classification and detection, while being competitive on word similarity tasks. It also outperforms previous systems on extrinsic classification tasks that benefit from exploiting lexical relational cues. We perform a series of analyses to understand the behaviors of our model. 1 Code available at https://github.com/aishikchakraborty/LexSub .
2020-11-30
Transactions of the Association for Computational Linguistics (publié)
doi.org
Recommandations pratiques pour une utilisation responsable de l’intelligence artificielle en santé mentale en contexte de pandémie
Carl-Maria Mörch
Pascale Lehoux
Marc-Antoine Dilhac
Catherine Régis
Xavier Dionne
La pandémie actuelle a provoqué une onde de choc dont les conséquences se font sentir dans tous les aspects de notre vie. Alors que la sa… (voir plus)nté physique a été généralement au cœur de l’attention scientifique et politique, il est devenu clair que la pandémie de COVID-19 a influé significativement sur la santé mentale de nombreux individus. Plus encore, elle aurait accentué les fragilités déjà existantes dans nos systèmes de santé mentale. Souvent moins financé ou soutenu que la santé physique, le domaine de la santé mentale pourrait-il bénéficier d’innovations en intelligence artificielle en période de pandémie ? Et si oui comment ? Que vous soyez développeur.e.s en IA, chercheur.e.s ou entrepreneur.e.s, ce document vise à vous fournir une synthèse des pistes d’actions et des ressources pour prévenir les principaux risques éthiques liés au développement d’applications d’IA dans le champ de la santé mentale. Pour illustrer ces principes, ce document propose de découvrir quatre cas fictif, à visée réaliste, à partir desquels il vous sera proposé de porter attention aux enjeux éthiques potentiels dans cette situation, aux enjeux d’innovation responsable à envisager, aux pistes d’action possibles inspirées de la liste de contrôle (Protocole Canadien conçu pour favoriser une utilisation responsable de l’IA en santé mentale et prévention du suicide, Mörch et al., 2020), aux ressources pratiques et à certains enjeux juridiques pertinents. Ce document a été élaboré par Carl-Maria Mörch, PhD, Algora Lab, Université de Montréal, Observatoire International sur les impacts sociétaux de l’Intelligence Artificielle et du Numérique (OBVIA), Mila – Institut Québécois d’Intelligence Artificielle, avec les contributions de Pascale Lehoux, Marc-Antoine Dilhac, Catherine Régis et Xavier Dionne.
2020-11-30
(publié)
doi.org
The default network of the human brain is associated with perceived social isolation
R. Nathan Spreng
Emile Dimas
Laetitia Mwilambwe-Tshilobo
Alain Dagher
Philipp Koellinger
Gideon Nave
Anthony Ong
Julius M. Kernbach
Thomas V. Wiecki
Tian Ge
Yue Li
Avram J. Holmes
B. T. Thomas Yeo
Gary R. Turner
Robin I. M. Dunbar
Danilo Bzdok
Humans survive and thrive through social exchange. Yet, social dependency also comes at a cost. Perceived social isolation, or loneliness, a… (voir plus)ffects physical and mental health, cognitive performance, overall life expectancy, and increases vulnerability to Alzheimer’s disease-related dementias. Despite severe consequences on behavior and health, the neural basis of loneliness remains elusive. Using the UK Biobank population imaging-genetics cohort (n = ~40,000, aged 40–69 years when recruited, mean age = 54.9), we test for signatures of loneliness in grey matter morphology, intrinsic functional coupling, and fiber tract microstructure. The loneliness-linked neurobiological profiles converge on a collection of brain regions known as the ‘default network’. This higher associative network shows more consistent loneliness associations in grey matter volume than other cortical brain networks. Lonely individuals display stronger functional communication in the default network, and greater microstructural integrity of its fornix pathway. The findings fit with the possibility that the up-regulation of these neural circuits supports mentalizing, reminiscence and imagination to fill the social void.
2020-11-30
Nature Communications (publié)
doi.org
Probabilistic Classifiers
Irina Rish
2020-11-29
Data-Driven Computational Neuroscience (publié)
doi.org
Universal Semantics for the Stochastic Lambda-Calculus
Pedro H. Azevedo de Amorim
Dexter Kozen
Radu Mardare
Prakash Panangaden
Michael Roberts
We define sound and adequate denotational and operational semantics for the stochastic lambda calculus. These two semantic approaches build … (voir plus)on previous work that used similar techniques to reason about higher-order probabilistic programs, but for the first time admit an adequacy theorem relating the operational and denotational views. This resolves the main issue left open in (Bacci et al. 2018).
2020-11-28
arXiv.org (prépublication)
dblp.uni-trier.de
#EEGManyLabs: Investigating the replicability of influential EEG experiments
Yuri G Pavlov
Nika Adamian
Stefan Appelhoff
Mahnaz Arvaneh
Christopher Benwell
Christian Beste
Amy Bland
Daniel E. Bradford
Florian Bublatzky
Niko A. Busch
Peter E. Clayson
Damian Cruse
Artur Czeszumski
Anna Dreber
Guillaume Dumas
Benedikt Valerian Ehinger
Giorgio Ganis
Xun He
José Antonio Hinojosa
Christoph Huber-Huber … (voir 39 de plus)
Michael Inzlicht
Bradley N Jack
Magnus Johannesson
Rhiannon Jones
Evgenii Kalenkovich
Laura Kaltwasser
Hamid Karimi-Rouzbahani
Andreas Keil
Peter König
Layla Kouara
Louisa Kulke
Cecile D. Ladouceur
Nicolas Langer
Heinrich R. Liesefeld
David Luque
Annmarie MacNamara
Liad Mudrik
Muthuraman Muthuraman
Lauren Browning Neal
Gustav Nilsonne
Guiomar Niso
Sebastian Ocklenburg
Robert Oostenveld
Cyril R. Pernet
Gilles Pourtois
Manuela Ruzzoli
Sarah Sass
Alexandre Schaefer
Magdalena Senderecka
Joel S. Snyder
Christian K. Tamnes
E Tognoli
Marieke K. van Vugt
Edelyn Verona
Robin Vloeberghs
Dominik Welke
Jan Wessel
Ilya Zakharov
Faisal Mushtaq
2020-11-26
Cortex (publié)
doi.org
Human attachments shape interbrain synchrony toward efficient performance of social goals
Amir Djalovski
Guillaume Dumas
Sivan Kinreich
Ruth Pinkenson Feldman
2020-11-25
NeuroImage (publié)
doi.org
Interactive Psychometrics for Autism With the Human Dynamic Clamp: Interpersonal Synchrony From Sensorimotor to Sociocognitive Domains
Florence Baillin
Aline Lefebvre
Amandine Pedoux
Yann Beauxis
Denis-Alexander Engemann
Anna Maruani
Frederique Amsellem
J. A. Scott Kelso
Thomas Bourgeron
Richard Delorme
Guillaume Dumas
The human dynamic clamp (HDC) is a human–machine interface designed on the basis of coordination dynamics for studying realistic social in… (voir plus)teraction under controlled and reproducible conditions. Here, we propose to probe the validity of the HDC as a psychometric instrument for quantifying social abilities in children with autism spectrum disorder (ASD) and neurotypical development. To study interpersonal synchrony with the HDC, we derived five standardized scores following a gradient from sensorimotor and motor to higher sociocognitive skills in a sample of 155 individuals (113 participants with ASD, 42 typically developing participants; aged 5 to 25 years; IQ > 70). Regression analyses were performed using normative modeling on global scores according to four subconditions (HDC behavior “cooperative/competitive,” human task “in-phase/anti-phase,” diagnosis, and age at inclusion). Children with ASD had lower scores than controls for motor skills. HDC motor coordination scores were the best candidates for stratification and diagnostic biomarkers according to exploratory analyses of hierarchical clustering and multivariate classification. Independently of phenotype, sociocognitive skills increased with developmental age while being affected by the ongoing task and HDC behavior. Weaker performance in ASD for motor skills suggests the convergent validity of the HDC for evaluating social interaction. Results provided additional evidence of a relationship between sensorimotor and sociocognitive skills. HDC may also be used as a marker of maturation of sociocognitive skills during real-time social interaction. Through its standardized and objective evaluation, the HDC not only represents a valid paradigm for the study of interpersonal synchrony but also offers a promising, clinically relevant psychometric instrument for the evaluation and stratification of sociomotor dysfunctions.
2020-11-25
Frontiers in Psychiatry (publié)
doi.org
RetroGNN: Approximating Retrosynthesis by Graph Neural Networks for De Novo Drug Design
Cheng-Hao Liu
Maksym Korablyov
Stanisław Jastrzębski
Paweł Włodarczyk-Pruszyński
Yoshua Bengio
Marwin Segler
De novo molecule generation often results in chemically unfeasible molecules. A natural idea to mitigate this problem is to bias the search … (voir plus)process towards more easily synthesizable molecules using a proxy for synthetic accessibility. However, using currently available proxies still results in highly unrealistic compounds. We investigate the feasibility of training deep graph neural networks to approximate the outputs of a retrosynthesis planning software, and their use to bias the search process. We evaluate our method on a benchmark involving searching for drug-like molecules with antibiotic properties. Compared to enumerating over five million existing molecules from the ZINC database, our approach finds molecules predicted to be more likely to be antibiotics while maintaining good drug-like properties and being easily synthesizable. Importantly, our deep neural network can successfully filter out hard to synthesize molecules while achieving a
2020-11-24
ArXiv (prépublication)
arxiv.org
arxiv.org
AR-DAE: Towards Unbiased Neural Entropy Gradient Estimation
Jae Hyun Lim
Aaron Courville
Christopher Pal
Chin-wei Huang
Entropy is ubiquitous in machine learning, but it is in general intractable to compute the entropy of the distribution of an arbitrary conti… (voir plus)nuous random variable. In this paper, we propose the amortized residual denoising autoencoder (AR-DAE) to approximate the gradient of the log density function, which can be used to estimate the gradient of entropy. Amortization allows us to significantly reduce the error of the gradient approximator by approaching asymptotic optimality of a regular DAE, in which case the estimation is in theory unbiased. We conduct theoretical and experimental analyses on the approximation error of the proposed method, as well as extensive studies on heuristics to ensure its robustness. Finally, using the proposed gradient approximator to estimate the gradient of entropy, we demonstrate state-of-the-art performance on density estimation with variational autoencoders and continuous control with soft actor-critic.
2020-11-20
Proceedings of the 37th International Conference on Machine Learning (publié)
doi.org
proceedings.mlr.press
Countering Language Drift with Seeded Iterated Learning
Yuchen Lu
Soumye Singhal
Florian Strub
Olivier Pietquin
Aaron Courville
Pretraining on human corpus and then finetuning in a simulator has become a standard pipeline for training a goal-oriented dialogue agent. N… (voir plus)evertheless, as soon as the agents are finetuned to maximize task completion, they suffer from the so-called language drift phenomenon: they slowly lose syntactic and semantic properties of language as they only focus on solving the task. In this paper, we propose a generic approach to counter language drift called Seeded iterated learning (SIL). We periodically refine a pretrained student agent by imitating data sampled from a newly generated teacher agent. At each time step, the teacher is created by copying the student agent, before being finetuned to maximize task completion. SIL does not require external syntactic constraint nor semantic knowledge, making it a valuable task-agnostic finetuning protocol. We evaluate SIL in a toy-setting Lewis Game, and then scale it up to the translation game with natural language. In both settings, SIL helps counter language drift as well as it improves the task completion compared to baselines.
2020-11-20
Proceedings of the 37th International Conference on Machine Learning (publié)
doi.org
proceedings.mlr.press