Publications

Inferring global-scale temporal latent topics from news reports to predict public health interventions for COVID-19
Zhi Wen
Guido Powell
Imane Chafi
David L. Buckeridge
Yue Li
The COVID-19 pandemic has highlighted the importance of non-pharmacological interventions (NPI) for controlling epidemics of emerging infect… (see more)ious diseases. Despite their importance, NPI have been monitored mainly through the manual efforts of volunteers. This approach hinders measurement of the NPI effectiveness and development of evidence to guide their use to control the global pandemic. We present EpiTopics, a machine learning approach to support automation of the NPI prediction and monitoring at both the document-level and country-level by mining the vast amount of unlabelled news reports on COVID-19. EpiTopics uses a 3-stage, transfer-learning algorithm to classify documents according to NPI categories, relying on topic modelling to support result interpretation. We identified 25 interpretable topics under 4 distinct and coherent COVID-related themes. Importantly, the use of these topics resulted in significant improvements over alternative automated methods in predicting the NPIs in labelled documents and in predicting country-level NPIs for 42 countries.
2021-12-31
Patterns (published)
doi.org
Interpolation Consistency Training for Semi-Supervised Learning
Vikas Verma
Kenji Kawaguchi
Alex Lamb
Juho Kannala
Yoshua Bengio
David Lopez-Paz
Arno Solin
2021-12-31
Neural Networks (published)
doi.org
arxiv.org
Investigating the Performance of Transformer-Based NLI Models on Presuppositional Inferences
Jad Kabbara
Jackie CK Cheung
Presuppositions are assumptions that are taken for granted by an utterance, and identifying them is key to a pragmatic interpretation of lan… (see more)guage. In this paper, we investigate the capabilities of transformer models to perform NLI on cases involving presupposition. First, we present simple heuristics to create alternative “contrastive” test cases based on the ImpPres dataset and investigate the model performance on those test cases. Second, to better understand how the model is making its predictions, we analyze samples from sub-datasets of ImpPres and examine model performance on them. Overall, our findings suggest that NLI-trained transformer models seem to be exploiting specific structural and lexical cues as opposed to performing some kind of pragmatic reasoning.
2021-12-31
COLING (published)
dblp.uni-trier.de
Joint Selection using Deep Reinforcement Learning for Skeleton-based Activity Recognition
Bahareh Nikpour
Narges Armanfard
<div>Skeleton based human activity recognition has attracted lots of attention due to its wide range of applications. Skeleton data in… (see more)cludes two or three dimensional coordinates of body joints. All of the body joints are not effective in recognizing different activities, so finding key joints within a video and across different activities has a significant role in improving the performance. In this paper we propose a novel framework that performs joint selection in skeleton video frames for the purpose of human activity recognition. To this end, we formulate the joint selection problem as a Markov Decision Process (MDP) where we employ deep reinforcement learning to find the most informative joints per frame. The proposed joint selection method is a general framework that can be employed to improve human activity classification methods. Experimental results on two benchmark activity recognition data sets using three different classifiers demonstrate effectiveness of the proposed joint selection method.</div>
2021-12-31
IEEE International Conference on Systems, Man, and Cybernetics (unknown)
doi.org
KNIFE: Kernelized-Neural Differential Entropy Estimation
Georg Pichler
Pierre Colombo
Malik Boudiaf
Gunther Koliander
Pablo Piantanida
Mutual Information (MI) has been widely used as a loss regularizer for training neural networks. This has been particularly effective when l… (see more)earn dis-entangled or compressed representations of high dimensional data. However, differential entropy (DE), another fundamental measure of information, has not found widespread use in neural network training. Although DE offers a potentially wider range of applications than MI, off-the-shelf DE estimators are either non differentiable, computationally intractable or fail to adapt to changes in the underlying distribution. These drawbacks prevent them from being used as regularizers in neural networks training. To address shortcomings in previously proposed estimators for DE, here we introduce K NIFE , a fully parameterized, differentiable kernel-based estimator of DE. The flexibility of our approach also allows us to construct K NIFE -based estimators for conditional (on either discrete or continuous variables) DE, as well as MI. We empirically validate our method on high-dimensional synthetic data and further apply it to guide the training of neural networks for real-world tasks. Our experiments on a large variety of tasks, including visual domain adaptation, textual fair classification, and textual fine-tuning demonstrate the effectiveness of K NIFE - based estimation. Code can be found at https: //github.com/g-pichler/knife .
2021-12-31
arXiv.org (preprint)
dblp.uni-trier.de
LAGr: Label Aligned Graphs for Better Systematic Generalization in Semantic Parsing
Dora Jambor
Dzmitry Bahdanau
Semantic parsing is the task of producing a structured meaning representation for natural language utterances or questions. Recent research … (see more)has pointed out that the commonly-used sequence-to-sequence (seq2seq) semantic parsers struggle to generalize systematically, i.e. to handle examples that require recombining known knowledge in novel settings. In this work, we show that better systematic generalization can be achieved by producing the meaning representation (MR) directly as a graph and not as a sequence. To this end we propose LAGr, the Labeling Aligned Graphs algorithm that produces semantic parses by predicting node and edge labels for a complete multi-layer input-aligned graph. The strongly-supervised LAGr algorithm requires aligned graphs as inputs, whereas weakly-supervised LAGr infers alignments for originally unaligned target graphs using an approximate MAP inference procedure. On the COGS and CFQ compositional generalization benchmarks the strongly- and weakly- supervised LAGr algorithms achieve significant improvements upon the baseline seq2seq parsers.
2021-12-31
Annual Meeting of the Association for Computational Linguistics (published)
doi.org
arxiv.org
Learning to evoke complex motor outputs with spatiotemporal neurostimulation using a hierarchical and adaptive optimization algorithm.
Samuel Laferrière
Marco Bonizzato
Numa Dancause
Guillaume Lajoie
The development of neurostimulation techniques to evoke motor patterns is an active area of research. It serves as a crucial experimental to… (see more)ol to probe computation in neural circuits, and it has applications in neuroprostheses used to aid recovery of function after stroke or injury. There are two important challenges when designing algorithms to unveil and control neurostimulation-to-motor mappings, thereby linking spatiotemporal patterns of neural stimulation to muscle activation: (1) the exploration of motor maps needs to be fast and efficient (exhaustive search is to be avoided for clinical and experimental reasons) (2) online learning needs to be flexible enough to track ongoing changes in these maps. We propose a stimulation search algorithm to address these issues, and demonstrate its efficacy with experiments in non-human primate models. Our solution is a novel iterative process using Bayesian Optimization via Gaussian Processes on increasingly complex signal spaces. We show that our algorithm can successfully and rapidly learn mappings between complex stimulation patterns and evoked muscle activation patterns, where standard approaches fail. Importantly, we uncover nonlinear circuit-level computations in M1 that would not have been possible to identify using conventional mapping techniques.
2021-12-31
Conference on Cognitive Computational Neuroscience (published)
doi.org
Learning Inter-Modal Correspondence and Phenotypes From Multi-Modal Electronic Health Records
Kejing Yin
William K. Cheung
Benjamin C. M. Fung
Jonathan Poon
Non-negative tensor factorization has been shown a practical solution to automatically discover phenotypes from the electronic health record… (see more)s (EHR) with minimal human supervision. Such methods generally require an input tensor describing the inter-modal interactions to be pre-established; however, the correspondence between different modalities (e.g., correspondence between medications and diagnoses) can often be missing in practice. Although heuristic methods can be applied to estimate them, they inevitably introduce errors, and leads to sub-optimal phenotype quality. This is particularly important for patients with complex health conditions (e.g., in critical care) as multiple diagnoses and medications are simultaneously present in the records. To alleviate this problem and discover phenotypes from EHR with unobserved inter-modal correspondence, we propose the collective hidden interaction tensor factorization (cHITF) to infer the correspondence between multiple modalities jointly with the phenotype discovery. We assume that the observed matrix for each modality is marginalization of the unobserved inter-modal correspondence, which are reconstructed by maximizing the likelihood of the observed matrices. Extensive experiments conducted on the real-world MIMIC-III dataset demonstrate that cHITF effectively infers clinically meaningful inter-modal correspondence, discovers phenotypes that are more clinically relevant and diverse, and achieves better predictive performance compared with a number of state-of-the-art computational phenotyping models.
2021-12-31
IEEE Transactions on Knowledge and Data Engineering (published)
doi.org
arxiv.org
Learning Representations for New Sound Classes With Continual Self-Supervised Learning
Zhepei Wang
Cem Subakan
Xilin Jiang
Junkai Wu
Efthymios Tzinis
Mirco Ravanelli
Paris Smaragdis
In this paper, we work on a sound recognition system that continually incorporates new sound classes. Our main goal is to develop a framewor… (see more)k where the model can be updated without relying on labeled data. For this purpose, we propose adopting representation learning, where an encoder is trained using unlabeled data. This learning framework enables the study and implementation of a practically relevant use case where only a small amount of the labels is available in a continual learning context. We also make the empirical observation that a similarity-based representation learning method within this framework is robust to forgetting even if no explicit mechanism against forgetting is employed. We show that this approach obtains similar performance compared to several distillation-based continual learning methods when employed on self-supervised representation learning methods.
2021-12-31
IEEE Signal Processing Letters (published)
doi.org
arxiv.org
Local Structure Matters Most in Most Languages
Louis Clouatre
Prasanna Parthasarathi
Amal Zouaq
A. Chandar
2021-12-31
AACL/IJCNLP (2) (published)
doi.org
arxiv.org
Low-Rank Representation of Reinforcement Learning Policies
Bogdan Mazoure
Thang Doan
Tianyu Li
Vladimir Makarenkov
Joelle Pineau
Doina Precup
Guillaume Rabuseau
We propose a general framework for policy representation for reinforcement learning tasks. This framework involves finding a low-dimensional… (see more) embedding of the policy on a reproducing kernel Hilbert space (RKHS). The usage of RKHS based methods allows us to derive strong theoretical guarantees on the expected return of the reconstructed policy. Such guarantees are typically lacking in black-box models, but are very desirable in tasks requiring stability and convergence guarantees. We conduct several experiments on classic RL domains. The results confirm that the policies can be robustly represented in a low-dimensional space while the embedded policy incurs almost no decrease in returns.
2021-12-31
Journal of Artificial Intelligence Research (published)
doi.org
Meta-topologies define distinct anatomical classes of brain tumors linked to histology and survival
Julius M. Kernbach
Daniel Delev
Georg Neuloh
Hans Clusmann
Danilo Bzdok
Simon B. Eickhoff
Victor E. Staartjes
Flavio Vasella
Michael Weller
Luca Regli
Carlo Serra
Niklaus Krayenbühl
Kevin Akeret
Abstract Background The current WHO classification integrates histological and molecular features of brain tumors. The aim of this study was… (see more) to identify generalizable topological patterns with the potential to add an anatomical dimension to the classification of brain tumors. Methods We applied non-negative matrix factorization as an unsupervised pattern discovery strategy to the fine-grained topographic tumor profiles of 936 patients with primary and secondary brain tumors. From the anatomical features alone, this machine learning algorithm enabled the extraction of latent topological tumor patterns, termed meta-topologies . The optimal parts-based representation was automatically determined in 10,000 split-half iterations. We further characterized each meta-topology’s unique histopathologic profile and survival probability, thus linking important biological and clinical information to the underlying anatomical patterns Results In primary brain tumors, six meta-topologies were extracted, each detailing a transpallial pattern with distinct parenchymal and ventricular compositions. We identified one infratentorial, one allopallial, three neopallial (parieto-occipital, frontal, temporal) and one unisegmental meta-topology. Each meta-topology mapped to distinct histopathologic and molecular profiles. The unisegmental meta-topology showed the strongest anatomical-clinical link demonstrating a survival advantage in histologically identical tumors. Brain metastases separated to an infra- and supratentorial meta-topology with anatomical patterns highlighting their affinity to the cortico-subcortical boundary of arterial watershed areas. Conclusions Using a novel data-driven approach, we identified generalizable topological patterns in both primary and secondary brain tumors Differences in the histopathologic profiles and prognosis of these anatomical tumor classes provide insights into the heterogeneity of tumor biology and might add to personalized clinical decision making.
2021-12-31
Brain and Spine (published)
doi.org