Publications

Coordinating Policies Among Multiple Agents via an Intelligent Communication Channel
Dianbo Liu
Vedant Shah
Oussama Boussif
Cristian Meo
Anirudh Goyal
Tianmin Shu
Michael Curtis Mozer
Nicolas Heess
Yoshua Bengio
In Multi-Agent Reinforcement Learning (MARL), specialized channels are often introduced that allow agents to communicate directly with one a… (see more)nother. In this paper, we propose an alternative approach whereby agents communicate through an intelligent facilitator that learns to sift through and interpret signals provided by all agents to improve the agents’ collective performance. To ensure that this facilitator does not become a centralized controller, agents are incentivized to reduce their dependence on the messages it conveys, and the messages can only influence the selection of a policy from a fixed set, not instantaneous actions given the policy. We demonstrate the strength of this architecture over existing baselines on several cooperative MARL environments.
2022-05-20
ArXiv (preprint)
doi.org
arxiv.org
Bayesian Structure Learning with Generative Flow Networks
Tristan Deleu
Antonio Gois
Chris Emezue
Mansi Rankawat
Simon Lacoste-Julien
Stefan Bauer
Yoshua Bengio
In Bayesian structure learning, we are interested in inferring a distribution over the directed acyclic graph (DAG) structure of Bayesian ne… (see more)tworks, from data. Defining such a distribution is very challenging, due to the combinatorially large sample space, and approximations based on MCMC are often required. Recently, a novel class of probabilistic models, called Generative Flow Networks (GFlowNets), have been introduced as a general framework for generative modeling of discrete and composite objects, such as graphs. In this work, we propose to use a GFlowNet as an alternative to MCMC for approximating the posterior distribution over the structure of Bayesian networks, given a dataset of observations. Generating a sample DAG from this approximate distribution is viewed as a sequential decision problem, where the graph is constructed one edge at a time, based on learned transition probabilities. Through evaluation on both simulated and real data, we show that our approach, called DAG-GFlowNet, provides an accurate approximation of the posterior over DAGs, and it compares favorably against other methods based on MCMC or variational inference.
2022-05-19
auai.org/UAI/2022/Conference (poster)
doi.org
proceedings.mlr.press
Hardware Architecture for Guessing Random Additive Noise Decoding Markov Order (GRAND-MO)
Syed Mohsin Abbas
Marwan Jalaleddine
Warren J. Gross
2022-05-19
Journal of Signal Processing Systems (published)
doi.org
Temporal Abstractions-Augmented Temporally Contrastive Learning: An Alternative to the Laplacian in RL
Akram Erraqabi
Marlos C. Machado
Harry Zhao
Mingde Zhao
Sainbayar Sukhbaatar
Alessandro Lazaric
Ludovic Denoyer
Yoshua Bengio
In reinforcement learning, the graph Laplacian has proved to be a valuable tool in the task-agnostic setting, with applications ranging from… (see more) skill discovery to reward shaping. Recently, learning the Laplacian representation has been framed as the optimization of a temporally-contrastive objective to overcome its computational limitations in large (or continuous) state spaces. However, this approach requires uniform access to all states in the state space, overlooking the exploration problem that emerges during the representation learning process. In this work, we propose an alternative method that is able to recover, in a non-uniform-prior setting, the expressiveness and the desired properties of the Laplacian representation. We do so by combining the representation learning with a skill-based covering policy, which provides a better training distribution to extend and refine the representation. We also show that a simple augmentation of the representation objective with the learned temporal abstractions improves dynamics-awareness and helps exploration. We find that our method succeeds as an alternative to the Laplacian in the non-uniform setting and scales to challenging continuous control environments. Finally, even if our method is not optimized for skill discovery, the learned skills can successfully solve difficult continuous navigation tasks with sparse rewards, where standard skill discovery approaches are no so effective.
2022-05-19
auai.org/UAI/2022/Conference (poster)
doi.org
proceedings.mlr.press
Universal antigen encoding of T cell activation from high-dimensional cytokine dynamics
Sooraj R. Achar
François X. P. Bourassa
Thomas J. Rademaker
Angela Lee
Taisuke Kondo
Emanuel Salazar-Cavazos
John S. Davies
Naomi Taylor
Paul François
Grégoire Altan-Bonnet
2022-05-19
Science (published)
doi.org
FedILC: Weighted Geometric Mean and Invariant Gradient Covariance for Federated Learning on Non-IID Data
Mike He Zhu
Lena Nehale Ezzine
Dianbo Liu
Yoshua Bengio
2022-05-18
ArXiv (preprint)
doi.org
arxiv.org
IL-flOw: Imitation Learning from Observation using Normalizing Flows
Wei-Di Chang
Juan Higuera
Scott Fujimoto
David Meger
Gregory Dudek
2022-05-18
ArXiv (preprint)
doi.org
arxiv.org
Human brain anatomy reflects separable genetic and environmental components of socioeconomic status
Hyeokmoon Kweon
Gökhan Aydogan
Alain Dagher
Danilo Bzdok
Christian C. Ruff
Gideon Nave
Martha J. Farah
Philipp D. Koellinger
Socioeconomic status (SES) correlates with brain structure, a relation of interest given the long-observed relations of SES to cognitive abi… (see more)lities and health. Yet, major questions remain open, in particular, the pattern of causality that underlies this relation. In an unprecedently large study, here, we assess genetic and environmental contributions to SES differences in neuroanatomy. We first establish robust SES–gray matter relations across a number of brain regions, cortical and subcortical. These regional correlates are parsed into predominantly genetic factors and those potentially due to the environment. We show that genetic effects are stronger in some areas (prefrontal cortex, insula) than others. In areas showing less genetic effect (cerebellum, lateral temporal), environmental factors are likely to be influential. Our results imply a complex interplay of genetic and environmental factors that influence the SES-brain relation and may eventually provide insights relevant to policy.
2022-05-17
Science Advances (published)
doi.org
Homogenization of SGD in high-dimensions: Exact dynamics and generalization properties
Courtney Paquette
Elliot Paquette
Ben Adlam
Jeffrey Pennington
2022-05-13
ArXiv (preprint)
arxiv.org
arxiv.org
Reconstruction of full-length LINE-1 progenitors from ancestral genomes
Laura F Campitelli
Isaac Yellan
Mihai Albu
Marjan Barazandeh
Zain M Patel
Mathieu Blanchette
Timothy R Hughes
2022-05-11
Genetics (published)
doi.org
Block Contextual MDPs for Continual Learning
Shagun Sodhani
Franziska Meier
Joelle Pineau
Amy Zhang
In reinforcement learning (RL), when defining a Markov Decision Process (MDP), the environment dynamics is implicitly assumed to be stationa… (see more)ry. This assumption of stationarity, while simplifying, can be unrealistic in many scenarios. In the continual reinforcement learning scenario, the sequence of tasks is another source of nonstationarity. In this work, we propose to examine this continual reinforcement learning setting through the Block Contextual MDP (BC-MDP) framework, which enables us to relax the assumption of stationarity. This framework challenges RL algorithms to handle both nonstationarity and rich observation settings and, by additionally leveraging smoothness properties, enables us to study generalization bounds for this setting. Finally, we take inspiration from adaptive control to propose a novel algorithm that addresses the challenges introduced by this more realistic BC-MDP setting, allows for zero-shot adaptation at evaluation time, and achieves strong performance on several nonstationary environments.
2022-05-10
Proceedings of The 4th Annual Learning for Dynamics and Control Conference (published)
proceedings.mlr.press
proceedings.mlr.press
Grow-and-Clip: Informative-yet-Concise Evidence Distillation for Answer Explanation
Yuyan Chen
Yanghua Xiao
Bang Liu
Interpreting the predictions of existing Question Answering (QA) models is critical to many real-world intelligent applications, such as QA … (see more)systems for healthcare, education, and finance. However, existing QA models lack interpretability and provide no feedback or explanation for end-users to help them understand why a specific prediction is the answer to a question. In this research, we argue that the evidences of an answer is critical to enhancing the interpretability of QA models. Unlike previous research that simply extracts several sentence(s) in the context as evidence, we are the first to explicitly define the concept of evidence as the supporting facts in a context which are informative, concise, and readable. Besides, we provide effective strategies to quantitatively measure the informativeness, conciseness and readability of evidence. Furthermore, we propose Grow-and-Clip Evidence Distillation (GCED) algorithm to extract evidences from the contexts by trade-off informativeness, conciseness, and readability. We conduct extensive experiments on the SQuAD and TriviaQA datasets with several baseline models to evaluate the effect of GCED on interpreting answers to questions. Human evaluation are also carried out to check the quality of distilled evidences. Experimental results show that automatic distilled evidences have human-like informativeness, conciseness and readability, which can enhance the interpretability of the answers to questions.
2022-05-08
2022 IEEE 38th International Conference on Data Engineering (ICDE) (published)
doi.org
arxiv.org