Publications

Evaluation of distortion correction methods in diffusion MRI of the spinal cord
Haykel Snoussi
Emmanuel Caruyer
Julien Cohen-Adad
Olivier Commowick
Benoit Combes
Élise Bannier
Anne Kerbrat
Christian Barillot
2021-08-08
ArXiv (preprint)
arxiv.org
arxiv.org
Learning function from structure in neuromorphic networks
Laura E. Suárez
Blake A. Richards
Guillaume Lajoie
Bratislav Misic
The connection patterns of neural circuits in the brain form a complex network. Collective signaling within the network manifests as pattern… (see more)ed neural activity, and is thought to support human cognition and adaptive behavior. Recent technological advances permit macro-scale reconstructions of biological brain networks. These maps, termed connectomes, display multiple non-random architectural features, including heavy-tailed degree distributions, segregated communities and a densely interconnected core. Yet, how computation and functional specialization emerge from network architecture remains unknown. Here we reconstruct human brain connectomes using in vivo diffusion-weighted imaging, and use reservoir computing to implement these connectomes as artificial neural networks. We then train these neuromorphic networks to learn a cognitive task. We show that biologically realistic neural architectures perform optimally when they display critical dynamics. We find that performance is driven by network topology, and that the modular organization of large-scale functional systems is computationally relevant. Throughout, we observe a prominent interaction between network structure and dynamics, such that the same underlying architecture can support a wide range of learning capacities across dynamical regimes. This work opens new opportunities to discover how the network organization of the brain optimizes cognitive capacity, conceptually bridging neuroscience and artificial intelligence.
2021-08-08
Nature Machine Intelligence (unknown)
doi.org
Neocortical inhibitory interneuron subtypes are differentially attuned to synchrony- and rate-coded information
Luke Y. Prince
Matthew M. Tran
Dorian Grey
Lydia Saad
Helen Chasiotis
Jeehyun Kwag
Michael M. Kohl
Blake A. Richards
Neurons can carry information with both the synchrony and rate of their spikes. However, it is unknown whether distinct subtypes of neurons … (see more)are more sensitive to information carried by synchrony versus rate, or vice versa. Here, we address this question using patterned optical stimulation in slices of somatosensory cortex from mouse lines labelling fast-spiking (FS) and regular-spiking (RS) interneurons. We used optical stimulation in layer 2/3 to encode a 1-bit signal using either the synchrony or rate of activity. We then examined the mutual information between this signal and the interneuron responses. We found that for a synchrony encoding, FS interneurons carried more information in the first five milliseconds, while both interneuron subtypes carried more information than excitatory neurons in later responses. For a rate encoding, we found that RS interneurons carried more information after several milliseconds. These data demonstrate that distinct interneuron subtypes in the neocortex have distinct sensitivities to synchrony versus rate codes.
2021-08-04
Communications Biology (published)
doi.org
Barriers and facilitators to patient engagement in patient safety from patients and healthcare professionals' perspectives: A systematic review and meta-synthesis.
Zahra Chegini
Morteza Arab‐Zozani
Sheikh Mohammed Shariful Islam
Georgia Tobiano
S. A. Rahimi
AIMS To explore patients' and healthcare professionals' (HCPs) perceived barriers and facilitators to patient engagement in patient safety. … (see more) METHODS We conducted a systematic review and meta-synthesis from five computerized databases, including PubMed/MEDLINE, Embase, Web of Science, Scopus and PsycINFO, as well as grey literature and reference lists of included studies. Data were last searched in December 2019 with no limitation on the year of publication. Qualitative and Mix-methods studies that explored HCPs' and patients' perceptions of barriers and facilitators to patient engagement in patient safety were included. Two authors independently screened the titles and the abstracts of studies. Next, the full texts of the screened studies were reviewed by two authors. Potential discrepancies were resolved by consensus with a third author. The Mixed Methods Appraisal Tool was used for quality appraisal. Thematic analysis was used to synthesize results. RESULTS Nineteen studies out of 2616 were included in this systematic review. Themes related to barriers included: patient unwillingness, HCPs' unwillingness, and inadequate infrastructures. Themes related to facilitators were: encouraging patients, sharing information with patients, establishing trustful relationship, establishing patient-centred care and improving organizational resources. CONCLUSION Patients have an active role in improving their safety. Strategies are required to address barriers that hinder or prevent patient engagement and create capacity and facilitate action.
2021-08-01
Nursing Forum (published)
doi.org
Sequoia: A Software Framework to Unify Continual Learning Research
Fabrice Normandin
Florian Golemo
Oleksiy Ostapenko
Pau Rodríguez
Matthew Riemer
J. Hurtado
Lucas Cecchi
Khimya Khetarpal
Dominic Zhao
Ryan Lindeborg
Timothee LESORT
David Vázquez
Laurent Charlin
Irina Rish
Massimo Caccia
The field of Continual Learning (CL) seeks to develop algorithms that accumulate knowledge and skills over time through interaction with non… (see more)-stationary environments. In practice, a plethora of evaluation procedures (settings) and algorithmic solutions (methods) exist, each with their own potentially disjoint set of assumptions. This variety makes measuring progress in CL difficult. We propose a taxonomy of settings, where each setting is described as a set of assumptions. A tree-shaped hierarchy emerges from this view, where more general settings become the parents of those with more restrictive assumptions. This makes it possible to use inheritance to share and reuse research, as developing a method for a given setting also makes it directly applicable onto any of its children. We instantiate this idea as a publicly available software framework called Sequoia, which features a wide variety of settings from both the Continual Supervised Learning (CSL) and Continual Reinforcement Learning (CRL) domains. Sequoia also includes a growing suite of methods which are easy to extend and customize, in addition to more specialized methods from external libraries. We hope that this new paradigm and its first implementation can help unify and accelerate research in CL. You can help us grow the tree by visiting www.github.com/lebrice/Sequoia.
2021-08-01
ArXiv (preprint)
doi.org
openreview.net
ADEPT: An Adjective-Dependent Plausibility Task
Ali Emami
Ian Porada
A.R. Olteanu
Kaheer Suleman
Adam Trischler
Jackie CK Cheung
2021-07-31
Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) (published)
doi.org
Guiding the Growth: Difficulty-Controllable Question Generation through Step-by-Step Rewriting
Yi Cheng
Siyao Li
Bang Liu
Ruihui Zhao
Sujian Li
Chenhua Lin
Yefeng Zheng
This paper explores the task of Difficulty-Controllable Question Generation (DCQG), which aims at generating questions with required difficu… (see more)lty levels. Previous research on this task mainly defines the difficulty of a question as whether it can be correctly answered by a Question Answering (QA) system, lacking interpretability and controllability. In our work, we redefine question difficulty as the number of inference steps required to answer it and argue that Question Generation (QG) systems should have stronger control over the logic of generated questions. To this end, we propose a novel framework that progressively increases question difficulty through step-by-step rewriting under the guidance of an extracted reasoning chain. A dataset is automatically constructed to facilitate the research, on which extensive experiments are conducted to test the performance of our method.
2021-07-31
Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) (published)
doi.org
arxiv.org
Integrating Semantics and Neighborhood Information with Graph-Driven Generative Models for Document Retrieval
Zijing Ou
Qinliang Su
Jianxing Yu
Bang Liu
Jingwen Wang
Ruihui Zhao
Changyou Chen
Yefeng Zheng
With the need of fast retrieval speed and small memory footprint, document hashing has been playing a crucial role in large-scale informatio… (see more)n retrieval. To generate high-quality hashing code, both semantics and neighborhood information are crucial. However, most existing methods leverage only one of them or simply combine them via some intuitive criteria, lacking a theoretical principle to guide the integration process. In this paper, we encode the neighborhood information with a graph-induced Gaussian distribution, and propose to integrate the two types of information with a graph-driven generative model. To deal with the complicated correlations among documents, we further propose a tree-structured approximation method for learning. Under the approximation, we prove that the training objective can be decomposed into terms involving only singleton or pairwise documents, enabling the model to be trained as efficiently as uncorrelated ones. Extensive experimental results on three benchmark datasets show that our method achieves superior performance over state-of-the-art methods, demonstrating the effectiveness of the proposed model for simultaneously preserving semantic and neighborhood information.
2021-07-31
Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) (published)
doi.org
arxiv.org
Minimax and Neyman-Pearson Meta-Learning for Outlier Languages
Edoardo Maria Ponti
Rahul Aralikatte
Disha Shrivastava
Siva Reddy
Anders Søgaard
Model-agnostic meta-learning (MAML) has been recently put forth as a strategy to learn resource-poor languages in a sample-efficient fashion… (see more). Nevertheless, the properties of these languages are often not well represented by those available during training. Hence, we argue that the i.i.d. assumption ingrained in MAML makes it ill-suited for cross-lingual NLP. In fact, under a decision-theoretic framework, MAML can be interpreted as minimising the expected risk across training languages (with a uniform prior), which is known as Bayes criterion. To increase its robustness to outlier languages, we create two variants of MAML based on alternative criteria: Minimax MAML reduces the maximum risk across languages, while Neyman-Pearson MAML constrains the risk in each language to a maximum threshold. Both criteria constitute fully differentiable two-player games. In light of this, we propose a new adaptive optimiser solving for a local approximation to their Nash equilibrium. We evaluate both model variants on two popular NLP tasks, part-of-speech tagging and question answering. We report gains for their average and minimum performance across low-resource languages in zero- and few-shot settings, compared to joint multi-source transfer and vanilla MAML.
2021-07-31
Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021 (published)
doi.org
arxiv.org
On-the-Fly Attention Modulation for Neural Generation
Yue Dong
Chandra Bhagavatula
Ximing Lu
Jena D. Hwang
Antoine Bosselut
Jackie CK Cheung
Yejin Choi
Despite considerable advancements with deep neural language models (LMs), neural text generation still suffers from degeneration: the genera… (see more)ted text is repetitive, generic, self-contradictory, and often lacks commonsense. Our analyses on sentence-level attention patterns in LMs reveal that neural degeneration may be associated with insufficient learning of task-specific characteristics by the attention mechanism. This finding motivates on-the-fly attention modulation -- a simple but effective method that enables the injection of priors into attention computation during inference. Automatic and human evaluation results on three text generation benchmarks demonstrate that attention modulation helps LMs generate text with enhanced fluency, creativity, and commonsense reasoning, in addition to significantly reduce sentence-level repetition.
2021-07-31
Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021 (published)
doi.org
arxiv.org
Optimizing Deeper Transformers on Small Datasets
Peng Xu
Dhruv Kumar
Wei Yang
Wenjie Zi
Keyi Tang
Chenyang Huang
Jackie Chi Kit Cheung
Simon J.D. Prince
Yanshuai Cao
It is a common belief that training deep transformers from scratch requires large datasets. Consequently, for small datasets, people usually… (see more) use shallow and simple additional layers on top of pre-trained models during fine-tuning. This work shows that this does not always need to be the case: with proper initialization and optimization, the benefits of very deep transformers can carry over to challenging tasks with small datasets, including Text-to-SQL semantic parsing and logical reading comprehension. In particular, we successfully train
2021-07-31
Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) (published)
doi.org
arxiv.org
Semantic and Syntactic Enhanced Aspect Sentiment Triplet Extraction
Zhexue Chen
Hong Huang
Bang Liu
Xuanhua Feng Shi
Hai-nan Jin
Aspect Sentiment Triplet Extraction (ASTE) aims to extract triplets from sentences, where each triplet includes an entity, its associated se… (see more)ntiment, and the opinion span explaining the reason for the sentiment. Most existing research addresses this problem in a multi-stage pipeline manner, which neglects the mutual information between such three elements and has the problem of error propagation. In this paper, we propose a Semantic and Syntactic Enhanced aspect Sentiment triplet Extraction model (S3E2) to fully exploit the syntactic and semantic relationships between the triplet elements and jointly extract them. Specifically, we design a Graph-Sequence duel representation and modeling paradigm for the task of ASTE: we represent the semantic and syntactic relationships between word pairs in a sentence by graph and encode it by Graph Neural Networks (GNNs), as well as modeling the original sentence by LSTM to preserve the sequential information. Under this setting, we further apply a more efficient inference strategy for the extraction of triplets. Extensive evaluations on four benchmark datasets show that S3E2 significantly outperforms existing approaches, which proves our S3E2's superiority and flexibility in an end-to-end fashion.
2021-07-31
Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021 (published)
doi.org
arxiv.org