Publications

Neuronal activity remodels the F-actin based submembrane lattice in dendrites but not axons of hippocampal neurons
Flavie Lavoie-Cardinal
Anthony Bilodeau
Mado Lemieux
Marc-André Gardner
Theresa Wiesner
Gabrielle Laramée
Christian Gagné
Paul De Koninck
2020-07-19
Scientific Reports (published)
doi.org
Survey on Applications of Multi-Armed and Contextual Bandits
Djallel Bouneffouf
Irina Rish
Charu Aggarwal
In recent years, the multi-armed bandit (MAB) framework has attracted a lot of attention in various applications, from recommender systems a… (see more)nd information retrieval to healthcare and finance. This success is due to its stellar performance combined with attractive properties, such as learning from less feedback. The multiarmed bandit field is currently experiencing a renaissance, as novel problem settings and algorithms motivated by various practical applications are being introduced, building on top of the classical bandit problem. This article aims to provide a comprehensive review of top recent developments in multiple real-life applications of the multi-armed bandit. Specifically, we introduce a taxonomy of common MAB-based applications and summarize the state-of-the-art for each of those domains. Furthermore, we identify important current trends and provide new perspectives pertaining to the future of this burgeoning field.
2020-07-18
2020 IEEE Congress on Evolutionary Computation (CEC) (published)
doi.org
Molecular signatures of cognition and affect
Justine Y. Hansen
Ross D. Markello
Jacob W. Vogel
Jakob Seidlitz
Danilo Bzdok
Bratislav Misic
Regulation of gene expression drives protein interactions that govern synaptic wiring and neuronal activity. The resulting coordinated activ… (see more)ity among neuronal populations supports complex psychological processes, yet how gene expression shapes cognition and emotion remains unknown. Here we directly bridge the microscale and macroscale by mapping gene expression patterns to functional activation patterns across the cortical sheet. Applying unsupervised learning to the Allen Human Brain Atlas and Neurosynth databases, we identify a ventromedial-dorsolateral gradient of gene assemblies that separate affective and cognitive domains. This topographic molecular-psychological signature reflects the hierarchical organization of the neocortex, including systematic variations in cell type, myeloarchitecture, laminar differentiation, and intrinsic network affiliation. In addition, this molecular-psychological signature is related to individual differences in cognitive performance, strengthens over neurodevelopment, and can be replicated in two independent repositories. Collectively, our results reveal spatially covarying transcriptomic and cognitive architectures, highlighting the influence that molecular mechanisms exert on psychological processes.
2020-07-15
bioRxiv (preprint)
doi.org
Learning to Navigate the Synthetically Accessible Chemical Space Using Reinforcement Learning
Sai Krishna Gottipati
Boris Sattarov
Sufeng Niu
Yashaswi Pathak
Haoran Wei
Shengchao Liu
Karam J. Thomas
Simon Blackburn
Connor W. Coley
Jian Tang
Sarath Chandar
Yoshua Bengio
Over the last decade, there has been significant progress in the field of machine learning for de novo drug design, particularly in deep gen… (see more)erative models. However, current generative approaches exhibit a significant challenge as they do not ensure that the proposed molecular structures can be feasibly synthesized nor do they provide the synthesis routes of the proposed small molecules, thereby seriously limiting their practical applicability. In this work, we propose a novel forward synthesis framework powered by reinforcement learning (RL) for de novo drug design, Policy Gradient for Forward Synthesis (PGFS), that addresses this challenge by embedding the concept of synthetic accessibility directly into the de novo drug design system. In this setup, the agent learns to navigate through the immense synthetically accessible chemical space by subjecting commercially available small molecule building blocks to valid chemical reactions at every time step of the iterative virtual multi-step synthesis process. The proposed environment for drug discovery provides a highly challenging test-bed for RL algorithms owing to the large state space and high-dimensional continuous action space with hierarchical actions. PGFS achieves state-of-the-art performance in generating structures with high QED and penalized clogP. Moreover, we validate PGFS in an in-silico proof-of-concept associated with three HIV targets. Finally, we describe how the end-to-end training conceptualized in this study represents an important paradigm in radically expanding the synthesizable chemical space and automating the drug discovery process.
2020-07-13
ICML (Accept)
doi.org
proceedings.mlr.press
Multi-Task Reinforcement Learning as a Hidden-Parameter Block MDP
Amy Zhang
Shagun Sodhani
Khimya Khetarpal
Joelle Pineau
Multi-task reinforcement learning is a rich paradigm where information from previously seen environments can be leveraged for better perform… (see more)ance and improved sample-efficiency in new environments. In this work, we leverage ideas of common structure underlying a family of Markov decision processes (MDPs) to improve performance in the few-shot regime. We use assumptions of structure from Hidden-Parameter MDPs and Block MDPs to propose a new framework, HiP-BMDP, and approach for learning a common representation and universal dynamics model. To this end, we provide transfer and generalization bounds based on task and state similarity, along with sample complexity bounds that depend on the aggregate number of samples across tasks, rather than the number of tasks, a significant improvement over prior work. To demonstrate the efficacy of the proposed method, we empirically compare and show improvements against other multi-task and meta-reinforcement learning baselines.
2020-07-13
ArXiv (preprint)
arxiv.org
arxiv.org
Stochastic Hamiltonian Gradient Methods for Smooth Games
Nicolas Loizou
Hugo Berard
Alexia Jolicoeur-Martineau
Pascal Vincent
Simon Lacoste-Julien
Ioannis Mitliagkas
The success of adversarial formulations in machine learning has brought renewed motivation for smooth games. In this work, we focus on the c… (see more)lass of stochastic Hamiltonian methods and provide the first convergence guarantees for certain classes of stochastic smooth games. We propose a novel unbiased estimator for the stochastic Hamiltonian gradient descent (SHGD) and highlight its benefits. Using tools from the optimization literature we show that SHGD converges linearly to the neighbourhood of a stationary point. To guarantee convergence to the exact solution, we analyze SHGD with a decreasing step-size and we also present the first stochastic variance reduced Hamiltonian method. Our results provide the first global non-asymptotic last-iterate convergence guarantees for the class of stochastic unconstrained bilinear games and for the more general class of stochastic games that satisfy a "sufficiently bilinear" condition, notably including some non-convex non-concave problems. We supplement our analysis with experiments on stochastic bilinear and sufficiently bilinear games, where our theory is shown to be tight, and on simple adversarial machine learning formulations.
2020-07-13
ICML (Accept)
doi.org
proceedings.mlr.press
On Variational Learning of Controllable Representations for Text without Supervision
Peng Xu
Jackie Chi Kit Cheung
Yanshuai Cao
The variational autoencoder (VAE) can learn the manifold of natural images on certain datasets, as evidenced by meaningful interpolating or … (see more)extrapolating in the continuous latent space. However, on discrete data such as text, it is unclear if unsupervised learning can discover similar latent space that allows controllable manipulation. In this work, we find that sequence VAEs trained on text fail to properly decode when the latent codes are manipulated, because the modified codes often land in holes or vacant regions in the aggregated posterior latent space, where the decoding network fails to generalize. Both as a validation of the explanation and as a fix to the problem, we propose to constrain the posterior mean to a learned probability simplex, and performs manipulation within this simplex. Our proposed method mitigates the latent vacancy problem and achieves the first success in unsupervised learning of controllable representations for text. Empirically, our method outperforms unsupervised baselines and strong supervised approaches on text style transfer, and is capable of performing more flexible fine-grained control over text generation than existing methods.
2020-07-13
ICML (Accept)
proceedings.mlr.press
proceedings.mlr.press
A Brief Look at Generalization in Visual Meta-Reinforcement Learning
Safa Alver
Doina Precup
Due to the realization that deep reinforcement learning algorithms trained on high-dimensional tasks can strongly overfit to their training … (see more)environments, there have been several studies that investigated the generalization performance of these algorithms. However, there has been no similar study that evaluated the generalization performance of algorithms that were specifically designed for generalization, i.e. meta-reinforcement learning algorithms. In this paper, we assess the generalization performance of these algorithms by leveraging high-dimensional, procedurally generated environments. We find that these algorithms can display strong overfitting when they are evaluated on challenging tasks. We also observe that scalability to high-dimensional tasks with sparse rewards remains a significant problem among many of the current meta-reinforcement learning algorithms. With these results, we highlight the need for developing meta-reinforcement learning algorithms that can both generalize and scale.
2020-07-12
ICML.cc/2020/Workshop/LifelongML (unknown)
openreview.net
openreview.net
Chaotic Continual Learning
Touraj Laleh
Mojtaba Faramarzi
Irina Rish
A. Chandar
Training a deep neural network requires the model to go over training data for several epochs and update network parameters. In continual le… (see more)arning, this process results in catastrophic forgetting which is one of the core issues of this domain. Most proposed approaches for this issue try to compensate for the effects of parameter updates in the batch incremental setup in which the training model visits a lot of samples for several epochs. However, it is not realistic to expect training data will always be fed to model in a batch incremental setup. This paper proposes a chaotic stream learner that mimics the chaotic behavior of biological neurons and does not updates network parameters. In addition, it can work with fewer samples compared to deep learning models on stream learning setup. Our experiments on MNIST, CIFAR10, and Omniglot show that the chaotic stream learner has less catastrophic forgetting by its nature in comparison to a CNN model in continual learning.
2020-07-12
ICML.cc/2020/Workshop/LifelongML (unknown)
openreview.net
openreview.net
Historical Issue Data of Projects on Jira
A. Nicholson
Deeksha M. Arya
Jin L.C. Guo
2020-07-12
(published)
doi.org
S2RMs: Spatially Structured Recurrent Modules
Nasim Rahaman
Anirudh Goyal
Muhammad Waleed Gondal
Manuel Wuthrich
Stefan Bauer
Y. Sharma
Yoshua Bengio
Bernhard Schölkopf
2020-07-12
ArXiv (preprint)
arxiv.org
arxiv.org
Towards an Unsupervised Method for Model Selection in Few-Shot Learning
Simon Guiroy
Vikas Verma
Christopher Pal
The study of generalization of neural networks in gradient-based meta-learning has recently great research interest. Previous work on the st… (see more)udy of the objective landscapes within the scope of few-shot classification empirically demonstrated that generalization to new tasks might be linked to the average inner product between their respective gradients vectors (Guiroy et al., 2019). Following that work, we study the effect that meta-training has on the learned space of representation of the network. Notably, we demonstrate that the global similarity in the space of representation, measured by the average inner product between the embeddings of meta-test examples, also correlates to generalization. Based on these observations, we propose a novel model-selection criterion for gradient-based meta-learning and experimentally validate its effectiveness.
2020-07-12
ICML.cc/2020/Workshop/LifelongML (unknown)
openreview.net
openreview.net