Publications

Continual Weight Updates and Convolutional Architectures for Equilibrium Propagation
Maxence Ernoult
Julie Grollier
Damien Querlioz
Yoshua Bengio
Benjamin Scellier
Equilibrium Propagation (EP) is a biologically inspired alternative algorithm to backpropagation (BP) for training neural networks. It appli… (see more)es to RNNs fed by a static input x that settle to a steady state, such as Hopfield networks. EP is similar to BP in that in the second phase of training, an error signal propagates backwards in the layers of the network, but contrary to BP, the learning rule of EP is spatially local. Nonetheless, EP suffers from two major limitations. On the one hand, due to its formulation in terms of real-time dynamics, EP entails long simulation times, which limits its applicability to practical tasks. On the other hand, the biological plausibility of EP is limited by the fact that its learning rule is not local in time: the synapse update is performed after the dynamics of the second phase have converged and requires information of the first phase that is no longer available physically. Our work addresses these two issues and aims at widening the spectrum of EP from standard machine learning models to more bio-realistic neural networks. First, we propose a discrete-time formulation of EP which enables to simplify equations, speed up training and extend EP to CNNs. Our CNN model achieves the best performance ever reported on MNIST with EP. Using the same discrete-time formulation, we introduce Continual Equilibrium Propagation (C-EP): the weights of the network are adjusted continually in the second phase of training using local information in space and time. We show that in the limit of slow changes of synaptic strengths and small nudging, C-EP is equivalent to BPTT (Theorem 1). We numerically demonstrate Theorem 1 and C-EP training on MNIST and generalize it to the bio-realistic situation of a neural network with asymmetric connections between neurons.
2020-04-28
ArXiv (preprint)
arxiv.org
arxiv.org
CNN to detect differences in cerebral cortical anatomy of left- and right- handers
Lisa Meyer-Baese
Erik Roecher
Lucas Moesch
Danilo Bzdok
Klaus Mathiak
Handedness is one of the most obvious functional asymmetries, but its relation to anatomical asymmetry in the brain has not yet been clearly… (see more) demonstrated. However, there is no significant evidence to prove or disprove this structure-function correlation, thus left-handed patients are often excluded from magnetic resonance imaging (MRI) studies. MRI classification of left and right hemispheres is a difficult task on its own due to the complexity of the images and the structural similarities between the two halves. We demonstrate a deep artificial neural network approach in connection with a detailed preprocessing pipeline for the classification of lateralization in T1-weighted MR images of the human brain. Preprocessing includes bias field correction and registration on the MNI template. Our classifier is a convolutional neural network (CNN) that was trained on 287 images. Each image was duplicated and mirrored on the mid-sagittal plane. The best model reached an accuracy of 97.594% with a mean of 95.42% and standard deviation of 1.37%. Additionally, our model’s performance was evaluated on an independent set of 118 images and reached a classification accuracy of 97%. In a larger study we tested the model on grey-matter images of 927 left and 927 right-handed patients from the UK Biobank. Here all right-handed images and all left-handed images were classified as belonging to one class. The results suggest that there is no structural difference in grey-matter between the two hemispheres that can be distinguished by the deep learning classifier.
2020-04-26
Real-Time Image Processing and Deep Learning 2020 (published)
doi.org
ArguLens: Anatomy of Community Opinions On Usability Issues Using Argumentation Models
Wenting Wang
Deeksha M. Arya
Nicole Novielli
Jinghui Cheng
Jin L.C. Guo
In open-source software (OSS), the design of usability is often influenced by the discussions among community members on platforms such as i… (see more)ssue tracking systems (ITSs). However, digesting the rich information embedded in issue discussions can be a major challenge due to the vast number and diversity of the comments. We propose and evaluate ArguLens, a conceptual framework and automated technique leveraging an argumentation model to support effective understanding and consolidation of community opinions in ITSs. Through content analysis, we anatomized highly discussed usability issues from a large, active OSS project, into their argumentation components and standpoints. We then experimented with supervised machine learning techniques for automated argument extraction. Finally, through a study with experienced ITS users, we show that the information provided by ArguLens supported the digestion of usability-related opinions and facilitated the review of lengthy issues. ArguLens provides the direction of designing valuable tools for high-level reasoning and effective discussion about usability.
2020-04-22
Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (published)
doi.org
arxiv.org
Inference for travel time on transportation networks
Mohamad Elmasri
Aurélie Labbe
Denis Larocque
Laurent Charlin
Travel time is essential for making travel decisions in real-world transportation networks. Understanding its distribution can resolve many … (see more)fundamental problems in transportation. Empirically, single-edge travel-time is well studied, but how to aggregate such information over many edges to arrive at the distribution of travel time over a route is still daunting. A range of statistical tools have been developed for network analysis; tools to study statistical behaviors of processes on dynamical networks are still lacking. This paper develops a novel statistical perspective to specific type of mixing ergodic processes (travel time), that mimic the behavior of travel time on real-world networks. Under general conditions on the single-edge speed (resistance) distribution, we show that travel time, normalized by distance, follows a Gaussian distribution with universal mean and variance parameters. We propose efficient inference methods for such parameters, and consequently asymptotic universal confidence and prediction intervals of travel time. We further develop path(route)-specific parameters that enable tighter Gaussian-based prediction intervals. We illustrate our methods with a real-world case study using mobile GPS data, where we show that the route-specific and universal intervals both achieve the 95\% theoretical coverage levels. Moreover, the route-specific prediction intervals result in tighter bounds that outperform competing models.
2020-04-22
(published)
www.semanticscholar.org
Prediction intervals for travel time on transportation networks
Mohamad Elmasri
Aurélie Labbe
Denis Larocque
Laurent Charlin
Estimating travel-time is essential for making travel decisions in transportation networks. Empirically, single road-segment travel-time is … (see more)well studied, but how to aggregate such information over many edges to arrive at the distribution of travel time over a route is still theoretically challenging. Understanding travel-time distribution can help resolve many fundamental problems in transportation, quantifying travel uncertainty as an example. We develop a novel statistical perspective to specific types of dynamical processes that mimic the behavior of travel time on real-world networks. We show that, under general conditions, travel-time normalized by distance, follows a Gaussian distribution with route-invariant (universal) location and scale parameters. We develop efficient inference methods for such parameters, with which we propose asymptotic universal confidence and prediction intervals of travel time. We further develop our theory to include road-segment level information to construct route-specific location and scale parameter sequences that produce tighter route-specific Gaussian-based prediction intervals. We illustrate our methods with a real-world case study using precollected mobile GPS data, where we show that the route-specific and route-invariant intervals both achieve the 95\% theoretical coverage levels, where the former result in tighter bounds that also outperform competing models.
2020-04-22
(published)
www.semanticscholar.org
Distinct roles of parvalbumin and somatostatin interneurons in gating the synchronization of spike times in the neocortex
Hyun Jae Jang
Hyowon Chung
James M. Rowland
Blake A. Richards
Michael M. Kohl
Jeehyun Kwag
Sensory information–driven spikes are synchronized across cortical layers by distinct subtypes of interneurons.
2020-04-21
Science Advances (published)
doi.org
Meta‐analytic evidence for a joint neural mechanism underlying response inhibition and state anger
Andrei A. Puiu
Olga Wudarczyk
Gregor Kohls
Danilo Bzdok
Beate Herpertz‐Dahlmann
Kerstin Konrad
Although anger may weaken response inhibition (RI) by allowing outbursts to bypass deliberate processing, it is equally likely that RI defic… (see more)its precipitate a state of anger (SA). In adolescents, for instance, anger occurs more frequently and often leads to escalating aggressive behaviors. Even though RI is considered a key component in explaining individual differences in SA expression, the neural overlap between SA and RI remains elusive. Here, we aimed to meta‐analytically revisit and update the neural correlates of motor RI, to determine a consistent neural architecture of SA, and to identify their joint neural network. Considering that inhibitory abilities follow a protracted maturation until early adulthood, we additionally computed RI meta‐analyses in youths and adults. Using activation likelihood estimation, we calculated twelve meta‐analyses across 157 RI and 39 SA experiments on healthy individuals. Consistent with previous findings, RI was associated with a broad frontoparietal network including the anterior insula/inferior frontal gyrus (aI/IFG), premotor and midcingulate cortices, extending into right temporoparietal areas. Youths showed convergent activity in right midcingulate and medial prefrontal areas, left aI/IFG, and the temporal poles. SA, on the other hand, reliably recruited the right aI/IFG and anterior cingulate cortex. Conjunction analyses between RI and SA yielded a single convergence cluster in the right aI/IFG. While frontoparietal networks and bilateral aI are ubiquitously recruited during RI, the right aI/IFG cluster likely represents a node in a dynamically‐adjusting monitoring network that integrates salient information thereby facilitating the execution of goal‐directed behaviors under highly unpredictable scenarios.
2020-04-20
Human Brain Mapping (published)
doi.org
To Write Code: The Cultural Fabrication of Programming Notation and Practice
Ian Arawjo
2020-04-20
International Conference on Human Factors in Computing Systems (published)
doi.org
Asking Questions the Human Way: Scalable Question-Answer Generation from Text Corpus
Bang Liu
Haojie Wei
Di Niu
Haolan Chen
Yancheng He
The ability to ask questions is important in both human and machine intelligence. Learning to ask questions helps knowledge acquisition, imp… (see more)roves question-answering and machine reading comprehension tasks, and helps a chatbot to keep the conversation flowing with a human. Existing question generation models are ineffective at generating a large amount of high-quality question-answer pairs from unstructured text, since given an answer and an input passage, question generation is inherently a one-to-many mapping. In this paper, we propose Answer-Clue-Style-aware Question Generation (ACS-QG), which aims at automatically generating high-quality and diverse question-answer pairs from unlabeled text corpus at scale by imitating the way a human asks questions. Our system consists of: i) an information extractor, which samples from the text multiple types of assistive information to guide question generation; ii) neural question generators, which generate diverse and controllable questions, leveraging the extracted assistive information; and iii) a neural quality controller, which removes low-quality generated data based on text entailment. We compare our question generation models with existing approaches and resort to voluntary human evaluation to assess the quality of the generated question-answer pairs. The evaluation results suggest that our system dramatically outperforms state-of-the-art neural question generation models in terms of the generation quality, while being scalable in the meantime. With models trained on a relatively smaller amount of data, we can generate 2.8 million quality-assured question-answer pairs from a million sentences found in Wikipedia.
2020-04-19
Proceedings of The Web Conference 2020 (published)
doi.org
arxiv.org
A Unifying Framework for Fairness-Aware Influence Maximization
Golnoosh Farnadi
Behrouz Babaki
Michel Gendreau
The problem of selecting a subset of nodes with greatest influence in a graph, commonly known as influence maximization, has been well studi… (see more)ed over the past decade. This problem has real world applications which can potentially affect lives of individuals. Algorithmic decision making in such domains raises concerns about their societal implications. One of these concerns, which surprisingly has only received limited attention so far, is algorithmic bias and fairness. We propose a flexible framework that extends and unifies the existing works in fairness-aware influence maximization. This framework is based on an integer programming formulation of the influence maximization problem. The fairness requirements are enforced by adding linear constraints or modifying the objective function. Contrary to the previous work which designs specific algorithms for each variant, we develop a formalism which is general enough for specifying different notions of fairness. A problem defined in this formalism can be then solved using efficient mixed integer programming solvers. The experimental evaluation indicates that our framework not only is general but also is competitive with existing algorithms.
2020-04-19
Companion Proceedings of the Web Conference 2020 (published)
doi.org
Role-Wise Data Augmentation for Knowledge Distillation
Jie Fu
Xue Geng
Zhijian Duan
Bohan Zhuang
Xingdi Yuan
Adam Trischler
Jie Lin
Vijay Chandrasekhar
Christopher Pal
Hao Dong
2020-04-18
ArXiv (preprint)
openreview.net
openreview.net
Neuropsychiatric copy number variants exert shared effects on human brain structure
Claudia Modenato
Kumar Kuldeep
Clara A. Moreau
Martin-Brevet Sandra
Huguet Guillaume
Schramm Catherine
Younis Nadine
Martin Charles-Olivier
C.O. Martin
Martineau Jean-Louis
Petra Tamer
Lippé Sarah
Thébault-Dagher Fanny
Valérie Côté
Charlebois A.R.
Deguire F.
Maillard Anne M.
Rodriguez-Herreros Borja
Pain Aurèlie
Richetin Sonia … (see 15 more)
16p11.2 European Consortium
Simons Variation in Individuals Project (VIP) Consortium
Kushan Leila
Silva Ana I.
Melie-Garcia Lester
Marianne B.M. van den Bree
Douard Elise
M. J. Owen
Hall Jeremy
Douard Elise
Chakravarty Mallar
Danilo Bzdok
Carrie E. Bearden
Draganski Bogdan
Sébastien Jacquemont
Neuropsychiatric CNVs share neuroanatomical signatures characterized by a parsimonious set of brain dimensions. The latter may underlie the … (see more)risk conferred by CNVs for a similar spectrum of neuropsychiatric conditions.
2020-04-16
medRxiv (preprint)
doi.org