Publications

Temporal Abstractions-Augmented Temporally Contrastive Learning: An Alternative to the Laplacian in RL
Akram Erraqabi
Marlos C. Machado
Mingde Zhao
Sainbayar Sukhbaatar
Alessandro Lazaric
Ludovic Denoyer
Yoshua Bengio
In reinforcement learning, the graph Laplacian has proved to be a valuable tool in the task-agnostic setting, with applications ranging from… (see more) skill discovery to reward shaping. Recently, learning the Laplacian representation has been framed as the optimization of a temporally-contrastive objective to overcome its computational limitations in large (or continuous) state spaces. However, this approach requires uniform access to all states in the state space, overlooking the exploration problem that emerges during the representation learning process. In this work, we propose an alternative method that is able to recover, in a non-uniform-prior setting, the expressiveness and the desired properties of the Laplacian representation. We do so by combining the representation learning with a skill-based covering policy, which provides a better training distribution to extend and refine the representation. We also show that a simple augmentation of the representation objective with the learned temporal abstractions improves dynamics-awareness and helps exploration. We find that our method succeeds as an alternative to the Laplacian in the non-uniform setting and scales to challenging continuous control environments. Finally, even if our method is not optimized for skill discovery, the learned skills can successfully solve difficult continuous navigation tasks with sparse rewards, where standard skill discovery approaches are no so effective.
2022-05-19
Conference on Uncertainty in Artificial Intelligence (poster)
doi.org
proceedings.mlr.press
Universal antigen encoding of T cell activation from high-dimensional cytokine dynamics
Sooraj R. Achar
François X. P. Bourassa
Thomas J. Rademaker
Angela Lee
Taisuke Kondo
Emanuel Salazar-Cavazos
John S. Davies
Naomi Taylor
Paul François
Grégoire Altan-Bonnet
2022-05-19
Science (published)
doi.org
FedILC: Weighted Geometric Mean and Invariant Gradient Covariance for Federated Learning on Non-IID Data
Mike He Zhu
Lena Nehale Ezzine
Dianbo Liu
Yoshua Bengio
2022-05-18
ArXiv (preprint)
doi.org
arxiv.org
IL-flOw: Imitation Learning from Observation using Normalizing Flows
Wei-Di Chang
Juan Higuera
Scott Fujimoto
David Meger
Gregory Dudek
2022-05-18
ArXiv (preprint)
doi.org
arxiv.org
Human brain anatomy reflects separable genetic and environmental components of socioeconomic status
Hyeokmoon Kweon
Gökhan Aydogan
Alain Dagher
Danilo Bzdok
Christian C. Ruff
Gideon Nave
Martha J. Farah
Philipp D. Koellinger
Socioeconomic status (SES) correlates with brain structure, a relation of interest given the long-observed relations of SES to cognitive abi… (see more)lities and health. Yet, major questions remain open, in particular, the pattern of causality that underlies this relation. In an unprecedently large study, here, we assess genetic and environmental contributions to SES differences in neuroanatomy. We first establish robust SES–gray matter relations across a number of brain regions, cortical and subcortical. These regional correlates are parsed into predominantly genetic factors and those potentially due to the environment. We show that genetic effects are stronger in some areas (prefrontal cortex, insula) than others. In areas showing less genetic effect (cerebellum, lateral temporal), environmental factors are likely to be influential. Our results imply a complex interplay of genetic and environmental factors that influence the SES-brain relation and may eventually provide insights relevant to policy.
2022-05-17
Science Advances (published)
doi.org
Homogenization of SGD in high-dimensions: Exact dynamics and generalization properties
Courtney Paquette
Elliot Paquette
Ben Adlam
Jeffrey Pennington
2022-05-13
ArXiv (preprint)
arxiv.org
arxiv.org
Myofiber reconstruction at micron scale reveals longitudinal bands in heart ventricular walls
Drisya Dileep
Tabish A. Syed
Tyler F. W. Sloan
Perundurai S. Dhandapany
Kaleem Siddiqi
Minhajuddin Sirajuddin
The coordinated contraction of myocytes drives the heart to beat and circulate blood. Due to the limited spatial resolution of whole heart i… (see more)maging and the piecemeal nature of high-magnification studies, a confirmed model of myofiber geometry does not yet exist. Using microscopy and computer vision we report the first three-dimensional reconstruction of myofibers across entire mouse ventricular walls at the micron scale, representing a gain of three orders of magnitude in spatial resolution over the existing models. Our analysis reveals prominent longitudinal bands of fibers that are orthogonal to the well-known circumferential ones. Our discovery impacts present understanding of heart wall mechanics and electrical function, with fundamental implications for the study of diseases related to myofiber disorganization.
2022-05-11
bioRxiv (preprint)
doi.org
Reconstruction of full-length LINE-1 progenitors from ancestral genomes
Laura F Campitelli
Isaac Yellan
Mihai Albu
Marjan Barazandeh
Zain M Patel
Mathieu Blanchette
Timothy R Hughes
2022-05-11
Genetics (published)
doi.org
Undiscounted Recursive Path Choice Models: Convergence Properties and Algorithms
Tien Mai
Emma Frejinger
2022-05-11
Transportation Science (published)
doi.org
Block Contextual MDPs for Continual Learning
Shagun Sodhani
Franziska Meier
Joelle Pineau
Amy Zhang
In reinforcement learning (RL), when defining a Markov Decision Process (MDP), the environment dynamics is implicitly assumed to be stationa… (see more)ry. This assumption of stationarity, while simplifying, can be unrealistic in many scenarios. In the continual reinforcement learning scenario, the sequence of tasks is another source of nonstationarity. In this work, we propose to examine this continual reinforcement learning setting through the Block Contextual MDP (BC-MDP) framework, which enables us to relax the assumption of stationarity. This framework challenges RL algorithms to handle both nonstationarity and rich observation settings and, by additionally leveraging smoothness properties, enables us to study generalization bounds for this setting. Finally, we take inspiration from adaptive control to propose a novel algorithm that addresses the challenges introduced by this more realistic BC-MDP setting, allows for zero-shot adaptation at evaluation time, and achieves strong performance on several nonstationary environments.
2022-05-10
Proceedings of The 4th Annual Learning for Dynamics and Control Conference (published)
proceedings.mlr.press
proceedings.mlr.press
Grow-and-Clip: Informative-yet-Concise Evidence Distillation for Answer Explanation
Yuyan Chen
Yanghua Xiao
Bang Liu
Interpreting the predictions of existing Question Answering (QA) models is critical to many real-world intelligent applications, such as QA … (see more)systems for healthcare, education, and finance. However, existing QA models lack interpretability and provide no feedback or explanation for end-users to help them understand why a specific prediction is the answer to a question. In this research, we argue that the evidences of an answer is critical to enhancing the interpretability of QA models. Unlike previous research that simply extracts several sentence(s) in the context as evidence, we are the first to explicitly define the concept of evidence as the supporting facts in a context which are informative, concise, and readable. Besides, we provide effective strategies to quantitatively measure the informativeness, conciseness and readability of evidence. Furthermore, we propose Grow-and-Clip Evidence Distillation (GCED) algorithm to extract evidences from the contexts by trade-off informativeness, conciseness, and readability. We conduct extensive experiments on the SQuAD and TriviaQA datasets with several baseline models to evaluate the effect of GCED on interpreting answers to questions. Human evaluation are also carried out to check the quality of distilled evidences. Experimental results show that automatic distilled evidences have human-like informativeness, conciseness and readability, which can enhance the interpretability of the answers to questions.
2022-05-08
2022 IEEE 38th International Conference on Data Engineering (ICDE) (published)
doi.org
arxiv.org
Learning What You Need from What You Did: Product Taxonomy Expansion with User Behaviors Supervision
Sijie Cheng
Zhouhong Gu
Bang Liu
Rui Xie
Wei Wu
Yanghua Xiao
Taxonomies have been widely used in various domains to underpin numerous applications. Specially, product taxonomies serve an essential role… (see more) in the e-commerce domain for the recommendation, browsing, and query understanding. However, taxonomies need to constantly capture the newly emerged terms or concepts in e-commerce platforms to keep up-to-date, which is expensive and labor-intensive if it relies on manual maintenance and updates. Therefore, we target the taxonomy expansion task to attach new concepts to existing taxonomies automatically. In this paper, we present a self-supervised and user behavior-oriented product taxonomy expansion framework to append new concepts into existing taxonomies. Our framework extracts hyponymy relations that conform to users' intentions and cognition. Specifically, i) to fully exploit user behavioral information, we extract candidate hyponymy relations that match user interests from query-click concepts; ii) to enhance the semantic information of new concepts and better detect hyponymy relations, we model concepts and relations through both user-generated content and structural information in existing taxonomies and user click logs, by leveraging Pre-trained Language Models and Graph Neural Network combined with Contrastive Learning; iii) to reduce the cost of dataset construction and overcome data skews, we construct a high-quality and balanced training dataset from existing taxonomy with no supervision. Extensive experiments on real-world product taxonomies in Meituan Platform, a leading Chinese vertical e-commerce platform to order take-out with more than 70 million daily active users, demonstrate the superiority of our proposed framework over state-of-the-art methods. Notably, our method enlarges the size of real-world product taxonomies from 39,263 to 94,698 relations with 88% precision. Our implementation is available: https://github.com/AdaCheng/Product_Taxonomy_Expansion.
2022-05-08
2022 IEEE 38th International Conference on Data Engineering (ICDE) (published)
doi.org
arxiv.org