We use cookies to analyze the browsing and usage of our website and to personalize your experience. You can disable these technologies at any time, but this may limit certain functionalities of the site. Read our Privacy Policy for more information.
Setting cookies
You can enable and disable the types of cookies you wish to accept. However certain choices you make could affect the services offered on our sites (e.g. suggestions, personalised ads, etc.).
Essential cookies
These cookies are necessary for the operation of the site and cannot be deactivated. (Still active)
Analytics cookies
Do you accept the use of cookies to measure the audience of our sites?
Multimedia Player
Do you accept the use of cookies to display and allow you to watch the video content hosted by our partners (YouTube, etc.)?
Publications
An Analysis of Dataset Overlap on Winograd-Style Tasks
The Winograd Schema Challenge (WSC) and variants inspired by it have become important benchmarks for common-sense reasoning (CSR). Model per… (see more)formance on the WSC has quickly progressed from chance-level to near-human using neural language models trained on massive corpora. In this paper, we analyze the effects of varying degrees of overlaps that occur between these corpora and the test instances in WSC-style tasks. We find that a large number of test instances overlap considerably with the pretraining corpora on which state-of-the-art models are trained, and that a significant drop in classification accuracy occurs when models are evaluated on instances with minimal overlap. Based on these results, we provide the WSC-Web dataset, consisting of over 60k pronoun disambiguation problems scraped from web data, being both the largest corpus to date, and having a significantly lower proportion of overlaps with current pretraining corpora.
2020-12-01
Proceedings of the 28th International Conference on Computational Linguistics (published)
Word embeddings are trained to predict word cooccurrence statistics, which leads them to possess different lexical properties (syntactic, se… (see more)mantic, etc.) depending on the notion of context defined at training time. These properties manifest when querying the embedding space for the most similar vectors, and when used at the input layer of deep neural networks trained to solve downstream NLP problems. Meta-embeddings combine multiple sets of differently trained word embeddings, and have been shown to successfully improve intrinsic and extrinsic performance over equivalent models which use just one set of source embeddings. We introduce word prisms: a simple and efficient meta-embedding method that learns to combine source embeddings according to the task at hand. Word prisms learn orthogonal transformations to linearly combine the input source embeddings, which allows them to be very efficient at inference time. We evaluate word prisms in comparison to other meta-embedding methods on six extrinsic evaluations and observe that word prisms offer improvements in performance on all tasks.
2020-12-01
Proceedings of the 28th International Conference on Computational Linguistics (published)
Abstract In this paper, we propose LexSub, a novel approach towards unifying lexical and distributional semantics. We inject knowledge about… (see more) lexical-semantic relations into distributional word embeddings by defining subspaces of the distributional vector space in which a lexical relation should hold. Our framework can handle symmetric attract and repel relations (e.g., synonymy and antonymy, respectively), as well as asymmetric relations (e.g., hypernymy and meronomy). In a suite of intrinsic benchmarks, we show that our model outperforms previous approaches on relatedness tasks and on hypernymy classification and detection, while being competitive on word similarity tasks. It also outperforms previous systems on extrinsic classification tasks that benefit from exploiting lexical relational cues. We perform a series of analyses to understand the behaviors of our model.1 Code available at https://github.com/aishikchakraborty/LexSub.
2020-12-01
Transactions of the Association for Computational Linguistics (published)
La pandémie actuelle a provoqué une onde de choc dont les conséquences se font sentir dans tous les aspects de notre vie. Alors que la sa… (see more)nté physique a été généralement au cœur de l’attention scientifique et politique, il est devenu clair que la pandémie de COVID-19 a influé significativement sur la santé mentale de nombreux individus. Plus encore, elle aurait accentué les fragilités déjà existantes dans nos systèmes de santé mentale. Souvent moins financé ou soutenu que la santé physique, le domaine de la santé mentale pourrait-il bénéficier d’innovations en intelligence artificielle en période de pandémie ? Et si oui comment ? Que vous soyez développeur.e.s en IA, chercheur.e.s ou entrepreneur.e.s, ce document vise à vous fournir une synthèse des pistes d’actions et des ressources pour prévenir les principaux risques éthiques liés au développement d’applications d’IA dans le champ de la santé mentale. Pour illustrer ces principes, ce document propose de découvrir quatre cas fictif, à visée réaliste, à partir desquels il vous sera proposé de porter attention aux enjeux éthiques potentiels dans cette situation, aux enjeux d’innovation responsable à envisager, aux pistes d’action possibles inspirées de la liste de contrôle (Protocole Canadien conçu pour favoriser une utilisation responsable de l’IA en santé mentale et prévention du suicide, Mörch et al., 2020), aux ressources pratiques et à certains enjeux juridiques pertinents. Ce document a été élaboré par Carl-Maria Mörch, PhD, Algora Lab, Université de Montréal, Observatoire International sur les impacts sociétaux de l’Intelligence Artificielle et du Numérique (OBVIA), Mila – Institut Québécois d’Intelligence Artificielle, avec les contributions de Pascale Lehoux, Marc-Antoine Dilhac, Catherine Régis et Xavier Dionne.
A fascinating hypothesis is that human and animal intelligence could be explained by a few principles (rather than an encyclopaedic list of … (see more)heuristics). If that hypothesis was correct, we could more easily both understand our own intelligence and build intelligent machines. Just like in physics, the principles themselves would not be sufficient to predict the behaviour of complex systems like brains, and substantial computation might be needed to simulate human-like intelligence. This hypothesis would suggest that studying the kind of inductive biases that humans and animals exploit could help both clarify these principles and provide inspiration for AI research and neuroscience theories. Deep learning already exploits several key inductive biases, and this work considers a larger list, focusing on those which concern mostly higher-level and sequential conscious processing. The objective of clarifying these particular principles is that they could potentially help us build AI systems benefiting from humans’ abilities in terms of flexible out-of-distribution and systematic generalization, which is currently an area where a large gap exists between state-of-the-art machine learning and human intelligence.
The human dynamic clamp (HDC) is a human–machine interface designed on the basis of coordination dynamics for studying realistic social in… (see more)teraction under controlled and reproducible conditions. Here, we propose to probe the validity of the HDC as a psychometric instrument for quantifying social abilities in children with autism spectrum disorder (ASD) and neurotypical development. To study interpersonal synchrony with the HDC, we derived five standardized scores following a gradient from sensorimotor and motor to higher sociocognitive skills in a sample of 155 individuals (113 participants with ASD, 42 typically developing participants; aged 5 to 25 years; IQ > 70). Regression analyses were performed using normative modeling on global scores according to four subconditions (HDC behavior “cooperative/competitive,” human task “in-phase/anti-phase,” diagnosis, and age at inclusion). Children with ASD had lower scores than controls for motor skills. HDC motor coordination scores were the best candidates for stratification and diagnostic biomarkers according to exploratory analyses of hierarchical clustering and multivariate classification. Independently of phenotype, sociocognitive skills increased with developmental age while being affected by the ongoing task and HDC behavior. Weaker performance in ASD for motor skills suggests the convergent validity of the HDC for evaluating social interaction. Results provided additional evidence of a relationship between sensorimotor and sociocognitive skills. HDC may also be used as a marker of maturation of sociocognitive skills during real-time social interaction. Through its standardized and objective evaluation, the HDC not only represents a valid paradigm for the study of interpersonal synchrony but also offers a promising, clinically relevant psychometric instrument for the evaluation and stratification of sociomotor dysfunctions.
De novo molecule generation often results in chemically unfeasible molecules. A natural idea to mitigate this problem is to bias the search … (see more)process towards more easily synthesizable molecules using a proxy for synthetic accessibility. However, using currently available proxies still results in highly unrealistic compounds. We investigate the feasibility of training deep graph neural networks to approximate the outputs of a retrosynthesis planning software, and their use to bias the search process. We evaluate our method on a benchmark involving searching for drug-like molecules with antibiotic properties. Compared to enumerating over five million existing molecules from the ZINC database, our approach finds molecules predicted to be more likely to be antibiotics while maintaining good drug-like properties and being easily synthesizable. Importantly, our deep neural network can successfully filter out hard to synthesize molecules while achieving a
Recent successes of game-theoretic formulations in ML have caused a resurgence of research interest in differentiable games. Overwhelmingly,… (see more) that research focuses on methods and upper bounds on their speed of convergence. In this work, we approach the question of fundamental iteration complexity by providing lower bounds to complement the linear (i.e. geometric) upper bounds observed in the literature on a wide class of problems. We cast saddle-point and min-max problems as 2-player games. We leverage tools from single-objective convex optimisation to propose new linear lower bounds for convex-concave games. Notably, we give a linear lower bound for
2020-11-21
Proceedings of the 37th International Conference on Machine Learning (published)
Experience replay is central to off-policy algorithms in deep reinforcement learning (RL), but there remain significant gaps in our understa… (see more)nding. We therefore present a systematic and extensive analysis of experience replay in Q-learning methods, focusing on two fundamental properties: the replay capacity and the ratio of learning updates to experience collected (replay ratio). Our additive and ablative studies upend conventional wisdom around experience replay -- greater capacity is found to substantially increase the performance of certain algorithms, while leaving others unaffected. Counterintuitively we show that theoretically ungrounded, uncorrected n-step returns are uniquely beneficial while other techniques confer limited benefit for sifting through larger memory. Separately, by directly controlling the replay ratio we contextualize previous observations in the literature and empirically measure its importance across a variety of deep RL algorithms. Finally, we conclude by testing a set of hypotheses on the nature of these performance benefits.
2020-11-21
Proceedings of the 37th International Conference on Machine Learning (published)
