Publications

A Deep Reinforcement Learning Approach to Marginalized Importance Sampling with the Successor Representation
Scott Fujimoto
David Meger
Doina Precup
Marginalized importance sampling (MIS), which measures the density ratio between the state-action occupancy of a target policy and that of a… (see more) sampling distribution, is a promising approach for off-policy evaluation. However, current state-of-the-art MIS methods rely on complex optimization tricks and succeed mostly on simple toy problems. We bridge the gap between MIS and deep reinforcement learning by observing that the density ratio can be computed from the successor representation of the target policy. The successor representation can be trained through deep reinforcement learning methodology and decouples the reward optimization from the dynamics of the environment, making the resulting algorithm stable and applicable to high-dimensional domains. We evaluate the empirical performance of our approach on a variety of challenging Atari and MuJoCo environments.
2021-06-30
Proceedings of the 38th International Conference on Machine Learning (published)
doi.org
proceedings.mlr.press
Directional Graph Networks: Anisotropic Aggregation in Graph Neural Networks via Directional Vector Fields
Dominique Beaini
Saro Passaro
Vincent Létourneau
William L. Hamilton
Gabriele Corso
Pietro Lio
The lack of anisotropic kernels in graph neural networks (GNNs) strongly limits their expressiveness, contributing to well-known issues such… (see more) as over-smoothing. To overcome this limitation, we propose the first globally consistent anisotropic kernels for GNNs, allowing for graph convolutions that are defined according to topologicaly-derived directional flows. First, by defining a vector field in the graph, we develop a method of applying directional derivatives and smoothing by projecting node-specific messages into the field. Then, we propose the use of the Laplacian eigenvectors as such vector field. We show that the method generalizes CNNs on an
2021-06-30
Proceedings of the 38th International Conference on Machine Learning (published)
doi.org
proceedings.mlr.press
Educating the future generation of researchers: A cross-disciplinary survey of trends in analysis methods
Taylor Bolt
Jason S. Nomi
Danilo Bzdok
Lucina Q. Uddin
Methods for data analysis in the biomedical, life, and social (BLS) sciences are developing at a rapid pace. At the same time, there is incr… (see more)easing concern that education in quantitative methods is failing to adequately prepare students for contemporary research. These trends have led to calls for educational reform to undergraduate and graduate quantitative research method curricula. We argue that such reform should be based on data-driven insights into within- and cross-disciplinary use of analytic methods. Our survey of peer-reviewed literature analyzed approximately 1.3 million openly available research articles to monitor the cross-disciplinary mentions of analytic methods in the past decade. We applied data-driven text mining analyses to the “Methods” and “Results” sections of a large subset of this corpus to identify trends in analytic method mentions shared across disciplines, as well as those unique to each discipline. We found that the t test, analysis of variance (ANOVA), linear regression, chi-squared test, and other classical statistical methods have been and remain the most mentioned analytic methods in biomedical, life science, and social science research articles. However, mentions of these methods have declined as a percentage of the published literature between 2009 and 2020. On the other hand, multivariate statistical and machine learning approaches, such as artificial neural networks (ANNs), have seen a significant increase in the total share of scientific publications. We also found unique groupings of analytic methods associated with each BLS science discipline, such as the use of structural equation modeling (SEM) in psychology, survival models in oncology, and manifold learning in ecology. We discuss the implications of these findings for education in statistics and research methods, as well as within- and cross-disciplinary collaboration.
2021-06-30
PLoS Biology (published)
doi.org
Equivariant Networks for Pixelized Spheres
Mehran Shakerinava
Siamak Ravanbakhsh
Pixelizations of Platonic solids such as the cube and icosahedron have been widely used to represent spherical data, from climate records to… (see more) Cosmic Microwave Background maps. Platonic solids have well-known global symmetries. Once we pixelize each face of the solid, each face also possesses its own local symmetries in the form of Euclidean isometries. One way to combine these symmetries is through a hierarchy. However, this approach does not adequately model the interplay between the two levels of symmetry transformations. We show how to model this interplay using ideas from group theory, identify the equivariant linear maps, and introduce equivariant padding that respects these symmetries. Deep networks that use these maps as their building blocks generalize gauge equivariant CNNs on pixelized spheres. These deep networks achieve state-of-the-art results on semantic segmentation for climate data and omnidirectional image processing. Code is available at https://git.io/JGiZA.
2021-06-30
Proceedings of the 38th International Conference on Machine Learning (published)
doi.org
proceedings.mlr.press
Locally Persistent Exploration in Continuous Control Tasks with Sparse Rewards
Susan Amin
Maziar Gomrokchi
Hossein Aboutalebi
Harsh Satija
Doina Precup
A major challenge in reinforcement learning is the design of exploration strategies, especially for environments with sparse reward structur… (see more)es and continuous state and action spaces. Intuitively, if the reinforcement signal is very scarce, the agent should rely on some form of short-term memory in order to cover its environment efficiently. We propose a new exploration method, based on two intuitions: (1) the choice of the next exploratory action should depend not only on the (Markovian) state of the environment, but also on the agent's trajectory so far, and (2) the agent should utilize a measure of spread in the state space to avoid getting stuck in a small region. Our method leverages concepts often used in statistical physics to provide explanations for the behavior of simplified (polymer) chains in order to generate persistent (locally self-avoiding) trajectories in state space. We discuss the theoretical properties of locally self-avoiding walks and their ability to provide a kind of short-term memory through a decaying temporal correlation within the trajectory. We provide empirical evaluations of our approach in a simulated 2D navigation task, as well as higher-dimensional MuJoCo continuous control locomotion tasks with sparse rewards.
2021-06-30
Proceedings of the 38th International Conference on Machine Learning (published)
doi.org
proceedings.mlr.press
Out-of-Distribution Generalization via Risk Extrapolation
David Krueger
Ethan Caballero
Joern-Henrik Jacobsen
Amy Zhang
Jonathan Binas
Dinghuai Zhang
Rémi Le Priol
Aaron Courville
Distributional shift is one of the major obstacles when transferring machine learning prediction systems from the lab to the real world. To … (see more)tackle this problem, we assume that variation across training domains is representative of the variation we might encounter at test time, but also that shifts at test time may be more extreme in magnitude. In particular, we show that reducing differences in risk across training domains can reduce a model's sensitivity to a wide range of extreme distributional shifts, including the challenging setting where the input contains both causal and anti-causal elements. We motivate this approach, Risk Extrapolation (REx), as a form of robust optimization over a perturbation set of extrapolated domains (MM-REx), and propose a penalty on the variance of training risks (V-REx) as a simpler variant. We prove that variants of REx can recover the causal mechanisms of the targets, while also providing some robustness to changes in the input distribution ("covariate shift"). By appropriately trading-off robustness to causally induced distributional shifts and covariate shift, REx is able to outperform alternative methods such as Invariant Risk Minimization in situations where these types of shift co-occur.
2021-06-30
Proceedings of the 38th International Conference on Machine Learning (published)
doi.org
proceedings.mlr.press
RNN with Particle Flow for Probabilistic Spatio-temporal Forecasting
Soumyasundar Pal
Liheng Ma
Yingxue Zhang
Mark J. Coates
Spatio-temporal forecasting has numerous applications in analyzing wireless, traffic, and financial networks. Many classical statistical mod… (see more)els often fall short in handling the complexity and high non-linearity present in time-series data. Recent advances in deep learning allow for better modelling of spatial and temporal dependencies. While most of these models focus on obtaining accurate point forecasts, they do not characterize the prediction uncertainty. In this work, we consider the time-series data as a random realization from a nonlinear state-space model and target Bayesian inference of the hidden states for probabilistic forecasting. We use particle flow as the tool for approximating the posterior distribution of the states, as it is shown to be highly effective in complex, high-dimensional settings. Thorough experimentation on several real world time-series datasets demonstrates that our approach provides better characterization of uncertainty while maintaining comparable accuracy to the state-of-the art point forecasting methods.
2021-06-30
Proceedings of the 38th International Conference on Machine Learning (published)
proceedings.mlr.press
proceedings.mlr.press
Smart About Meds (SAM): a pilot randomized controlled trial of a mobile application to improve medication adherence following hospital discharge
Bettina Habib
David Buckeridge
Melissa Bustillo
Santiago Nicolas Marquez
Manish Thakur
Thai Tran
Daniala L. Weir
Robyn Tamblyn
The objectives of this pilot study were (1) to assess the feasibility of a larger evaluation of Smart About Meds (SAM), a patient-centered m… (see more)edication management mobile application, and (2) to evaluate SAM’s potential to improve outcomes of interest, including adherence to medication changes made at hospital discharge and the occurrence of adverse events. We conducted a pilot randomized controlled trial among patients discharged from internal medicine units of an academic health center between June 2019 and March 2020. Block randomization was used to randomize patients to intervention (received access to SAM at discharge) or control (received usual care). Patients were followed for 30 days post-discharge, during which app use was recorded. Pharmacy claims data were used to measure adherence to medication changes made at discharge, and physician billing data were used to identify emergency department visits and hospital readmissions during follow-up. Forty-nine patients were eligible for inclusion in the study at hospital discharge (23 intervention, 26 control). In the 30 days of post-discharge, 15 (65.2%) intervention patients used the SAM app. During this period, intervention patients adhered to a larger proportion of medication changes (83.7%) than control patients (77.8%), including newly prescribed medications (72.7% vs 61.7%) and dose changes (90.9% vs 81.8%). A smaller proportion of intervention patients (8.7%) were readmitted to hospital during follow-up than control patients (15.4%). The high uptake of SAM among intervention patients supports the feasibility of a larger trial. Results also suggest that SAM has the potential to enhance adherence to medication changes and reduce the risk of downstream adverse events. This hypothesis needs to be tested in a larger trial. Clinicaltrials.gov, registration number NCT04676165.
2021-06-30
JAMIA Open (published)
doi.org
Measures of balance in combinatorial optimization
Philippe Olivier
Andrea Lodi
Gilles Pesant
2021-06-24
4OR (published)
doi.org
Deep learning for AI
Yoshua Bengio
Yann Lecun
Geoffrey Hinton
How can neural networks learn the rich internal representations required for difficult tasks such as recognizing objects or understanding la… (see more)nguage?
2021-06-20
Communications of the ACM (published)
doi.org
Large-Scale Intrinsic Functional Brain Organization Emerges from Three Canonical Spatiotemporal Patterns
Taylor Bolt
Jason S. Nomi
Danilo Bzdok
Catie Chang
B.T. Yeo
Lucina Q. Uddin
Shella Keilholz
2021-06-19
(published)
doi.org
Digitizing a sustainable future
Lucia A. Reisch
Lucas Joppa
Peter Howson
Artur Gil
Panayiota Alevizou
Nina Michaelidou
Ruby Appiah-Campbell
Tilman Santarius
Susanne Köhler
Massimo Pizzol
Pia-Johanna Schweizer
Dipti Srinivasan
Lynn H. Kaack
Priya L. Donti
David Rolnick
2021-06-17
One Earth (published)
doi.org