Publications

Optimal Local and Remote Controllers With Unreliable Uplink Channels: An Elementary Proof
Mohammad Afshari
Aditya Mahajan
Recently, a model of a decentralized control system with local and remote controllers connected over unreliable channels was presented in [… (voir plus)1]. The model has a nonclassical information structure that is not partially nested. Nonetheless, it is shown in [1] that the optimal control strategies are linear functions of the state estimate (which is a nonlinear function of the observations). Their proof is based on a fairly sophisticated dynamic programming argument. In this article, we present an alternative and elementary proof of the result which uses common information-based conditional independence and completion of squares.
2020-07-31
IEEE Transactions on Automatic Control (publié)
doi.org
arxiv.org
Renewal Monte Carlo: Renewal Theory-Based Reinforcement Learning
Jayakumar Subramanian
Aditya Mahajan
An online reinforcement learning algorithm called renewal Monte Carlo (RMC) is presented. RMC works for infinite horizon Markov decision pro… (voir plus)cesses with a designated start state. RMC is a Monte Carlo algorithm that retains the key advantages of Monte Carlo—viz., simplicity, ease of implementation, and low bias—while circumventing the main drawbacks of Monte Carlo—viz., high variance and delayed updates. Given a parameterized policy
2020-07-31
IEEE Transactions on Automatic Control (publié)
doi.org
Towards Queryable and Traceable Domain Models
Rijul Saini
Gunter Mussbacher
Jin L.C. Guo
Jörg Kienzle
Model-Driven Software Engineering encompasses various modelling formalisms for supporting software development. One such formalism is domain… (voir plus) modelling which bridges the gap between requirements expressed in natural language and analyzable and more concise domain models expressed in class diagrams. Due to the lack of modelling skills among novice modellers and time constraints in industrial projects, it is often not possible to build an accurate domain model manually. To address this challenge, we aim to develop an approach to extract domain models from problem descriptions written in natural language by combining rules based on natural language processing with machine learning. As a first step, we report on an automated and tool-supported approach with an accuracy of extracted domain models higher than existing approaches. In addition, the approach generates trace links for each model element of a domain model. The trace links enable novice modellers to execute queries on the extracted domain models to gain insights into the modelling decisions taken for improving their modelling skills. Furthermore, to evaluate our approach, we propose a novel comparison metric and discuss our experimental design. Finally, we present a research agenda detailing research directions and discuss corresponding challenges.
2020-07-31
IEEE International Requirements Engineering Conference (publié)
doi.org
Inferring disease subtypes from clusters in explanation space
Marc-Andre Schulz
Matt Chapman-Rounds
Manisha Verma
Danilo Bzdok
Konstantinos Georgatzis
Identification of disease subtypes and corresponding biomarkers can substantially improve clinical diagnosis and treatment selection. Discov… (voir plus)ering these subtypes in noisy, high dimensional biomedical data is often impossible for humans and challenging for machines. We introduce a new approach to facilitate the discovery of disease subtypes: Instead of analyzing the original data, we train a diagnostic classifier (healthy vs. diseased) and extract instance-wise explanations for the classifier’s decisions. The distribution of instances in the explanation space of our diagnostic classifier amplifies the different reasons for belonging to the same class–resulting in a representation that is uniquely useful for discovering latent subtypes. We compare our ability to recover subtypes via cluster analysis on model explanations to classical cluster analysis on the original data. In multiple datasets with known ground-truth subclasses, particularly on UK Biobank brain imaging data and transcriptome data from the Cancer Genome Atlas, we show that cluster analysis on model explanations substantially outperforms the classical approach. While we believe clustering in explanation space to be particularly valuable for inferring disease subtypes, the method is more general and applicable to any kind of sub-type identification.
2020-07-29
Scientific Reports (publié)
doi.org
Patterns of autism symptoms: hidden structure in the ADOS and ADI-R instruments
Jeremy Lefort-Besnard
Kai Vogeley
Leonhard Schilbach
Gael Varoquaux
Bertrand Thirion
Guillaume Dumas
Danilo Bzdok
2020-07-29
Translational Psychiatry (publié)
doi.org
Deriving Differential Target Propagation from Iterating Approximate Inverses
Yoshua Bengio
2020-07-28
ArXiv (prépublication)
arxiv.org
arxiv.org
Predicting COVID-19 Pneumonia Severity on Chest X-ray With Deep Learning
Joseph Paul Cohen
Lan Dao
Paul Morrison
Karsten Roth
Yoshua Bengio
Beiyi Shen
Almas F Abbasi
Hoshmand Kochi Mahsa
Marzyeh Ghassemi
Haifang Li
Tim Q Duong
Introduction The need to streamline patient management for coronavirus disease-19 (COVID-19) has become more pressing than ever. Chest X-ray… (voir plus)s (CXRs) provide a non-invasive (potentially bedside) tool to monitor the progression of the disease. In this study, we present a severity score prediction model for COVID-19 pneumonia for frontal chest X-ray images. Such a tool can gauge the severity of COVID-19 lung infections (and pneumonia in general) that can be used for escalation or de-escalation of care as well as monitoring treatment efficacy, especially in the ICU. Methods Images from a public COVID-19 database were scored retrospectively by three blinded experts in terms of the extent of lung involvement as well as the degree of opacity. A neural network model that was pre-trained on large (non-COVID-19) chest X-ray datasets is used to construct features for COVID-19 images which are predictive for our task. Results This study finds that training a regression model on a subset of the outputs from this pre-trained chest X-ray model predicts our geographic extent score (range 0-8) with 1.14 mean absolute error (MAE) and our lung opacity score (range 0-6) with 0.78 MAE. Conclusions These results indicate that our model’s ability to gauge the severity of COVID-19 lung infections could be used for escalation or de-escalation of care as well as monitoring treatment efficacy, especially in the ICU. To enable follow up work, we make our code, labels, and data available online.
2020-07-27
Cureus (publié)
doi.org
arxiv.org
Neuronal activity remodels the F-actin based submembrane lattice in dendrites but not axons of hippocampal neurons
Flavie Lavoie-Cardinal
Anthony Bilodeau
Mado Lemieux
Marc-André Gardner
Theresa Wiesner
Gabrielle Laramée
Christian Gagné
Paul De Koninck
2020-07-19
Scientific Reports (publié)
doi.org
Survey on Applications of Multi-Armed and Contextual Bandits
Djallel Bouneffouf
Irina Rish
Charu Aggarwal
In recent years, the multi-armed bandit (MAB) framework has attracted a lot of attention in various applications, from recommender systems a… (voir plus)nd information retrieval to healthcare and finance. This success is due to its stellar performance combined with attractive properties, such as learning from less feedback. The multiarmed bandit field is currently experiencing a renaissance, as novel problem settings and algorithms motivated by various practical applications are being introduced, building on top of the classical bandit problem. This article aims to provide a comprehensive review of top recent developments in multiple real-life applications of the multi-armed bandit. Specifically, we introduce a taxonomy of common MAB-based applications and summarize the state-of-the-art for each of those domains. Furthermore, we identify important current trends and provide new perspectives pertaining to the future of this burgeoning field.
2020-07-18
2020 IEEE Congress on Evolutionary Computation (CEC) (publié)
doi.org
Molecular signatures of cognition and affect
Justine Y. Hansen
Ross D. Markello
Jacob W. Vogel
Jakob Seidlitz
Danilo Bzdok
Bratislav Misic
Regulation of gene expression drives protein interactions that govern synaptic wiring and neuronal activity. The resulting coordinated activ… (voir plus)ity among neuronal populations supports complex psychological processes, yet how gene expression shapes cognition and emotion remains unknown. Here we directly bridge the microscale and macroscale by mapping gene expression patterns to functional activation patterns across the cortical sheet. Applying unsupervised learning to the Allen Human Brain Atlas and Neurosynth databases, we identify a ventromedial-dorsolateral gradient of gene assemblies that separate affective and cognitive domains. This topographic molecular-psychological signature reflects the hierarchical organization of the neocortex, including systematic variations in cell type, myeloarchitecture, laminar differentiation, and intrinsic network affiliation. In addition, this molecular-psychological signature is related to individual differences in cognitive performance, strengthens over neurodevelopment, and can be replicated in two independent repositories. Collectively, our results reveal spatially covarying transcriptomic and cognitive architectures, highlighting the influence that molecular mechanisms exert on psychological processes.
2020-07-15
bioRxiv (prépublication)
doi.org
Learning to Navigate the Synthetically Accessible Chemical Space Using Reinforcement Learning
Sai Krishna Gottipati
Boris Sattarov
Sufeng Niu
Yashaswi Pathak
Haoran Wei
Shengchao Liu
Karam J. Thomas
Simon Blackburn
Connor W. Coley
Jian Tang
Sarath Chandar
Yoshua Bengio
Over the last decade, there has been significant progress in the field of machine learning for de novo drug design, particularly in deep gen… (voir plus)erative models. However, current generative approaches exhibit a significant challenge as they do not ensure that the proposed molecular structures can be feasibly synthesized nor do they provide the synthesis routes of the proposed small molecules, thereby seriously limiting their practical applicability. In this work, we propose a novel forward synthesis framework powered by reinforcement learning (RL) for de novo drug design, Policy Gradient for Forward Synthesis (PGFS), that addresses this challenge by embedding the concept of synthetic accessibility directly into the de novo drug design system. In this setup, the agent learns to navigate through the immense synthetically accessible chemical space by subjecting commercially available small molecule building blocks to valid chemical reactions at every time step of the iterative virtual multi-step synthesis process. The proposed environment for drug discovery provides a highly challenging test-bed for RL algorithms owing to the large state space and high-dimensional continuous action space with hierarchical actions. PGFS achieves state-of-the-art performance in generating structures with high QED and penalized clogP. Moreover, we validate PGFS in an in-silico proof-of-concept associated with three HIV targets. Finally, we describe how the end-to-end training conceptualized in this study represents an important paradigm in radically expanding the synthesizable chemical space and automating the drug discovery process.
2020-07-13
International Conference on Machine Learning (Accept)
doi.org
proceedings.mlr.press
Multi-Task Reinforcement Learning as a Hidden-Parameter Block MDP
Amy Zhang
Shagun Sodhani
Khimya Khetarpal
Joelle Pineau
Multi-task reinforcement learning is a rich paradigm where information from previously seen environments can be leveraged for better perform… (voir plus)ance and improved sample-efficiency in new environments. In this work, we leverage ideas of common structure underlying a family of Markov decision processes (MDPs) to improve performance in the few-shot regime. We use assumptions of structure from Hidden-Parameter MDPs and Block MDPs to propose a new framework, HiP-BMDP, and approach for learning a common representation and universal dynamics model. To this end, we provide transfer and generalization bounds based on task and state similarity, along with sample complexity bounds that depend on the aggregate number of samples across tasks, rather than the number of tasks, a significant improvement over prior work. To demonstrate the efficacy of the proposed method, we empirically compare and show improvements against other multi-task and meta-reinforcement learning baselines.
2020-07-13
ArXiv (prépublication)
arxiv.org
arxiv.org