Publications

Phylogenetic Manifold Regularization: A semi-supervised approach to predict transcription factor binding sites
Faizy Ahsan
Alexandre Drouin
Francois Laviolette
Doina Precup
Mathieu Blanchette
The computational prediction of transcription factor binding sites remains a challenging problems in bioinformatics, despite significant m e… (voir plus)thodological d evelopments f rom t he field of machine learning. Such computational models are essential to help interpret the non-coding portion of human genomes, and to learn more about the regulatory mechanisms controlling gene expression. In parallel, massive genome sequencing efforts have produced assembled genomes for hundred of vertebrate species, but this data is underused. We present PhyloReg, a new semi-supervised learning approach that can be used for a wide variety of sequence-to-function prediction problems, and that takes advantage of hundreds of millions of years of evolution to regularize predictors and improve accuracy. We demonstrate that PhyloReg can be used to better train a previously proposed deep learning model of transcription factor binding. Simulation studies further help delineate the benefits o f t he a pproach. G ains in prediction accuracy are obtained over a broad set of transcription factors and cell types.
2020-12-15
2020 IEEE International Conference on Bioinformatics and Biomedicine (BIBM) (publié)
doi.org
Graph Neural Networks Learn Twitter Bot Behaviour
Albert M. Orozco Camacho
Sacha Lévy
Reihaneh Rabbany
Social media trends are increasingly taking a significant role for the understanding of modern social dynamics. In this work, we take a look… (voir plus) at how the Twitter landscape gets constantly shaped by automatically generated content. Twitter bot activity can be traced via network abstractions which, we hypothesize, can be learned through state-of-the-art graph neural network techniques. We employ a large bot database, continuously updated by Twitter, to learn how likely is that a user is mentioned by a bot, as well as, for a hashtag. Thus, we model this likelihood as a link prediction task between the set of users and hashtags. Moreover, we contrast our results by performing similar experiments on a crawled data set of real users.
2020-12-11
LatinX in AI at Neural Information Processing Systems Conference 2020 (publié)
doi.org
Machine Learning for Glacier Monitoring in the Hindu Kush Himalaya
Shimaa Baraka
Benjamin Akera
Bibek Aryal
Tenzing Chogyal Sherpa
Finu Shresta
Anthony Ortiz
Kris Sankaran
Juan Lavista Ferres
M. Matin
Yoshua Bengio
2020-12-08
ArXiv (prépublication)
arxiv.org
arxiv.org
Adversarial Soft Advantage Fitting: Imitation Learning without Policy Optimization
Paul Barde
Julien Roy
Wonseok Jeon
Joelle Pineau
Christopher Pal
Derek Nowrouzezahrai
Adversarial Imitation Learning alternates between learning a discriminator -- which tells apart expert's demonstrations from generated ones … (voir plus)-- and a generator's policy to produce trajectories that can fool this discriminator. This alternated optimization is known to be delicate in practice since it compounds unstable adversarial training with brittle and sample-inefficient reinforcement learning. We propose to remove the burden of the policy optimization steps by leveraging a novel discriminator formulation. Specifically, our discriminator is explicitly conditioned on two policies: the one from the previous generator's iteration and a learnable policy. When optimized, this discriminator directly learns the optimal generator's policy. Consequently, our discriminator's update solves the generator's optimization problem for free: learning a policy that imitates the expert does not require an additional optimization loop. This formulation effectively cuts by half the implementation and computational burden of Adversarial Imitation Learning algorithms by removing the Reinforcement Learning phase altogether. We show on a variety of tasks that our simpler approach is competitive to prevalent Imitation Learning methods.
2020-12-06
Conference on Neural Information Processing Systems (Accept (poster))
doi.org
arxiv.org
Differentiable Causal Discovery from Interventional Data
Philippe Brouillard
Sébastien Lachapelle
Alexandre Lacoste
Simon Lacoste-Julien
Alexandre Drouin
Learning a causal directed acyclic graph from data is a challenging task that involves solving a combinatorial problem for which the solutio… (voir plus)n is not always identifiable. A new line of work reformulates this problem as a continuous constrained optimization one, which is solved via the augmented Lagrangian method. However, most methods based on this idea do not make use of interventional data, which can significantly alleviate identifiability issues. This work constitutes a new step in this direction by proposing a theoretically-grounded method based on neural networks that can leverage interventional data. We illustrate the flexibility of the continuous-constrained framework by taking advantage of expressive neural architectures such as normalizing flows. We show that our approach compares favorably to the state of the art in a variety of settings, including perfect and imperfect interventions for which the targeted nodes may even be unknown.
2020-12-06
Conference on Neural Information Processing Systems (Accept (poster))
doi.org
arxiv.org
An Equivalence between Loss Functions and Non-Uniform Sampling in Experience Replay
Scott Fujimoto
David Meger
Doina Precup
Prioritized Experience Replay (PER) is a deep reinforcement learning technique in which agents learn from transitions sampled with non-unifo… (voir plus)rm probability proportionate to their temporal-difference error. We show that any loss function evaluated with non-uniformly sampled data can be transformed into another uniformly sampled loss function with the same expected gradient. Surprisingly, we find in some environments PER can be replaced entirely by this new loss function without impact to empirical performance. Furthermore, this relationship suggests a new branch of improvements to PER by correcting its uniformly sampled loss function equivalent. We demonstrate the effectiveness of our proposed modifications to PER and the equivalent loss function in several MuJoCo and Atari environments.
2020-12-06
Conference on Neural Information Processing Systems (Accept (poster))
doi.org
arxiv.org
Equivariant Maps for Hierarchical Structures
Renhao Wang
Marjan Albooyeh
Siamak Ravanbakhsh
2020-12-06
Conference on Neural Information Processing Systems (Accept)
dblp.uni-trier.de
Forethought and Hindsight in Credit Assignment
Veronica Chelu
Doina Precup
Hado van Hasselt
We address the problem of credit assignment in reinforcement learning and explore fundamental questions regarding the way in which an agent … (voir plus)can best use additional computation to propagate new information, by planning with internal models of the world to improve its predictions. Particularly, we work to understand the gains and peculiarities of planning employed as forethought via forward models or as hindsight operating with backward models. We establish the relative merits, limitations and complementary properties of both planning mechanisms in carefully constructed scenarios. Further, we investigate the best use of models in planning, primarily focusing on the selection of states in which predictions should be (re)-evaluated. Lastly, we discuss the issue of model estimation and highlight a spectrum of methods that stretch from explicit environment-dynamics predictors to more abstract planner-aware models.
2020-12-06
Conference on Neural Information Processing Systems (Accept (poster))
doi.org
arxiv.org
Hybrid Models for Learning to Branch
Prateek Gupta
Maxime Gasse
Elias B. Khalil
M. Pawan Kumar
Andrea Lodi
Yoshua Bengio
A recent Graph Neural Network (GNN) approach for learning to branch has been shown to successfully reduce the running time of branch-and-bou… (voir plus)nd algorithms for Mixed Integer Linear Programming (MILP). While the GNN relies on a GPU for inference, MILP solvers are purely CPU-based. This severely limits its application as many practitioners may not have access to high-end GPUs. In this work, we ask two key questions. First, in a more realistic setting where only a CPU is available, is the GNN model still competitive? Second, can we devise an alternate computationally inexpensive model that retains the predictive power of the GNN architecture? We answer the first question in the negative, and address the second question by proposing a new hybrid architecture for efficient branching on CPU machines. The proposed architecture combines the expressive power of GNNs with computationally inexpensive multi-layer perceptrons (MLP) for branching. We evaluate our methods on four classes of MILP problems, and show that they lead to up to 26% reduction in solver running time compared to state-of-the-art methods without a GPU, while extrapolating to harder problems than it was trained on. The code for this project is publicly available at https://github.com/pg2455/Hybrid-learn2branch.
2020-12-06
Conference on Neural Information Processing Systems (Accept (poster))
doi.org
arxiv.org
Measuring Systematic Generalization in Neural Proof Generation with Transformers
Nicolas Gontier
Koustuv Sinha
Siva Reddy
Christopher Pal
We are interested in understanding how well Transformer language models (TLMs) can perform reasoning tasks when trained on knowledge encoded… (voir plus) in the form of natural language. We investigate their systematic generalization abilities on a logical reasoning task in natural language, which involves reasoning over relationships between entities grounded in first-order logical proofs. Specifically, we perform soft theorem-proving by leveraging TLMs to generate natural language proofs. We test the generated proofs for logical consistency, along with the accuracy of the final inference. We observe length-generalization issues when evaluated on longer-than-trained sequences. However, we observe TLMs improve their generalization performance after being exposed to longer, exhaustive proofs. In addition, we discover that TLMs are able to generalize better using backward-chaining proofs compared to their forward-chaining counterparts, while they find it easier to generate forward chaining proofs. We observe that models that are not trained to generate proofs are better at generalizing to problems based on longer proofs. This suggests that Transformers have efficient internal reasoning strategies that are harder to interpret. These results highlight the systematic generalization behavior of TLMs in the context of logical reasoning, and we believe this work motivates deeper inspection of their underlying reasoning strategies.
2020-12-06
Conference on Neural Information Processing Systems (Accept (poster))
doi.org
arxiv.org
Online Fast Adaptation and Knowledge Accumulation (OSAKA): a New Approach to Continual Learning
Massimo Caccia
Pau Rodríguez
Oleksiy Ostapenko
Fabrice Normandin
Min Lin
Lucas Caccia
Issam Laradji
Irina Rish
Alexandre Lacoste
David Vázquez
Laurent Charlin
Continual learning studies agents that learn from streams of tasks without forgetting previous ones while adapting to new ones. Two recent c… (voir plus)ontinual-learning scenarios have opened new avenues of research. In meta-continual learning, the model is pre-trained to minimize catastrophic forgetting of previous tasks. In continual-meta learning, the aim is to train agents for faster remembering of previous tasks through adaptation. In their original formulations, both methods have limitations. We stand on their shoulders to propose a more general scenario, OSAKA, where an agent must quickly solve new (out-of-distribution) tasks, while also requiring fast remembering. We show that current continual learning, meta-learning, meta-continual learning, and continual-meta learning techniques fail in this new scenario. We propose Continual-MAML, an online extension of the popular MAML algorithm as a strong baseline for this scenario. We empirically show that Continual-MAML is better suited to the new scenario than the aforementioned methodologies, as well as standard continual learning and meta-learning approaches.
2020-12-06
Conference on Neural Information Processing Systems (Accept (poster))
doi.org
arxiv.org
Reward Propagation Using Graph Convolutional Networks
Martin Klissarov
Doina Precup
Potential-based reward shaping provides an approach for designing good reward functions, with the purpose of speeding up learning. However, … (voir plus)automatically finding potential functions for complex environments is a difficult problem (in fact, of the same difficulty as learning a value function from scratch). We propose a new framework for learning potential functions by leveraging ideas from graph representation learning. Our approach relies on Graph Convolutional Networks which we use as a key ingredient in combination with the probabilistic inference view of reinforcement learning. More precisely, we leverage Graph Convolutional Networks to perform message passing from rewarding states. The propagated messages can then be used as potential functions for reward shaping to accelerate learning. We verify empirically that our approach can achieve considerable improvements in both small and high-dimensional control problems.
2020-12-06
Conference on Neural Information Processing Systems (Accept (poster))
doi.org
arxiv.org