Publications

Robust Policy Learning over Multiple Uncertainty Sets
Annie Xie
Shagun Sodhani
Chelsea Finn
Joelle Pineau
Amy Zhang
Reinforcement learning (RL) agents need to be robust to variations in safety-critical environments. While system identification methods prov… (see more)ide a way to infer the variation from online experience, they can fail in settings where fast identification is not possible. Another dominant approach is robust RL which produces a policy that can handle worst-case scenarios, but these methods are generally designed to achieve robustness to a single uncertainty set that must be specified at train time. Towards a more general solution, we formulate the multi-set robustness problem to learn a policy robust to different perturbation sets. We then design an algorithm that enjoys the benefits of both system identification and robust RL: it reduces uncertainty where possible given a few interactions, but can still act robustly with respect to the remaining uncertainty. On a diverse set of control tasks, our approach demonstrates improved worst-case performance on new environments compared to prior methods based on system identification and on robust RL alone.
2022-02-14
ArXiv (preprint)
arxiv.org
arxiv.org
Erratum to: Rapid simultaneous acquisition of macromolecular tissue volume, susceptibility, and relaxometry maps (Magn Reson Med. 2022;87:781‐790.)
Fang Frank Yu
Susie Y. Huang
Ashwin S. Kumar
T. Witzel
Congyu Liao
Tanguy Duval
Julien Cohen-Adad
Berkin Bilgic
2022-02-10
Magnetic Resonance in Medicine (published)
doi.org
Erratum to: Rapid simultaneous acquisition of macromolecular tissue volume, susceptibility, and relaxometry maps (Magn Reson Med. 2022;87:781‐790.)
Fang Frank Yu
Susie Yi Huang
Ashwin Kumar
Thomas Witzel
Congyu Liao
Tanguy Duval
Julien Cohen-Adad
Berkin Bilgic
2022-02-10
Magnetic Resonance in Medicine (published)
doi.org
Multivariate, Transgenerational Associations of the COVID-19 Pandemic Across Minoritized and Marginalized Communities.
S. Yip
Ayana Jordan
Robert J. Kohler
Avram J. Holmes
Danilo Bzdok
Importance The experienced consequences of the COVID-19 pandemic have diverged across individuals, families, and communities, resulting in i… (see more)nequity within a host of factors. There is a gap of quantitative evidence about the transgenerational impacts of these experiences and factors. Objective To identify baseline predictors of COVID-19 experiences, as defined by child and parent report, using a multivariate pattern-learning framework from the Adolescent Brain and Cognitive Development (ABCD) cohort. Design, Setting, and Participants ABCD is an ongoing prospective longitudinal study of child and adolescent development in the United States including 11 875 youths, enrolled at age 9 to 10 years. Using nationally collected longitudinal profiling data from 9267 families, a multivariate pattern-learning strategy was developed to identify factor combinations associated with transgenerational costs of the ongoing COVID-19 pandemic. ABCD data (release 3.0) collected from 2016 to 2020 and released between 2019 and 2021 were analyzed in combination with ABCD COVID-19 rapid response data from the first 3 collection points (May-August 2020). Exposures Social distancing and other response measures imposed by COVID-19, including school closures and shutdown of many childhood recreational activities. Main Outcomes and Measures Mid-COVID-19 experiences as defined by the ABCD's parent and child COVID-19 assessments. Results Deep profiles from 9267 youth (5681 female [47.8%]; mean [SD] age, 119.0 [7.5] months) and their caregivers were quantitatively examined. Enabled by a pattern-learning analysis, social determinants of inequity, including family structure, socioeconomic status, and the experience of racism, were found to be primarily associated with transgenerational impacts of COVID-19, above and beyond other candidate predictors such as preexisting medical or psychiatric conditions. Pooling information across more than 17 000 baseline pre-COVID-19 family indicators and more than 280 measures of day-to-day COVID-19 experiences, non-White (ie, families who reported being Asian, Black, Hispanic, other, or a combination of those choices) and/or Spanish-speaking families were found to have decreased resources (mode 1, canonical vector weight [CVW] = 0.19; rank 5 of 281), escalated likelihoods of financial worry (mode 1, CVW = -0.20; rank 4), and food insecurity (mode 1, CVW = 0.21; rank 2), yet were more likely to have parent-child discussions regarding COVID-19-associated health and prevention issues, such as handwashing (mode 1, CVW = 0.14; rank 9), conserving food or other items (mode 1, CVW = 0.21; rank 1), protecting elderly individuals (mode 1, CVW = 0.11; rank 21), and isolating from others (mode 1, CVW = 0.11; rank 23). In contrast, White families (mode 1, CVW = -0.07; rank 3), those with higher pre-COVID-19 income (mode 1, CVW = -0.07; rank 5), and presence of a parent with a postgraduate degree (mode 1, CVW = -0.06; rank 14) experienced reduced COVID-19-associated impact. In turn, children from families experiencing reduced COVID-19 impacts reported longer nighttime sleep durations (mode 1, CVW = 0.13; rank 14), less difficulties with remote learning (mode 2, CVW = 0.14; rank 7), and decreased worry about the impact of COVID-19 on their family's financial stability (mode 1, CVW = 0.134; rank 13). Conclusions and Relevance The findings of this study indicate that community-level, transgenerational intervention strategies may be needed to combat the disproportionate burden of pandemics on minoritized and marginalized racial and ethnic populations.
2022-02-09
JAMA psychiatry (published)
doi.org
The Brain-Computer Metaphor Debate Is Useless: A Matter of Semantics
Blake Richards
Timothy P. Lillicrap
2022-02-08
Frontiers in Computer Science (published)
doi.org
On learning Whittle index policy for restless bandits with scalable regret
Nima Akbarzadeh
Aditya Mahajan
Reinforcement learning is an attractive approach to learn good resource allocation and scheduling policies based on data when the system mod… (see more)el is unknown. However, the cumulative regret of most RL algorithms scales as
2022-02-07
ArXiv (preprint)
doi.org
arxiv.org
Tackling Climate Change with Machine Learning
David Rolnick
Priya L. Donti
Lynn H. Kaack
Kelly Kochanski
Alexandre Lacoste
Kris Sankaran
Andrew Slavin Ross
Nikola Milojevic-Dupont
Natasha Jaques
Anna Waldman-Brown
Alexandra Luccioni
Tegan Maharaj
Evan David Sherwin
S. Karthik Mukkavilli
Konrad Paul Kording
Carla P. Gomes
Andrew Y. Ng
Demis Hassabis
John C. Platt
Felix Creutzig … (see 2 more)
Jennifer T Chayes
Yoshua Bengio
Climate change is one of the greatest challenges facing humanity, and we, as machine learning (ML) experts, may wonder how we can help. Here… (see more) we describe how ML can be a powerful tool in reducing greenhouse gas emissions and helping society adapt to a changing climate. From smart grids to disaster management, we identify high impact problems where existing gaps can be filled by ML, in collaboration with other fields. Our recommendations encompass exciting research questions as well as promising business opportunities. We call on the ML community to join the global effort against climate change.
2022-02-07
ACM Computing Surveys (published)
doi.org
arxiv.org
TACTiS: Transformer-Attentional Copulas for Time Series
Alexandre Drouin
Étienne Marcotte
Nicolas Chapados
The estimation of time-varying quantities is a fundamental component of decision making in fields such as healthcare and finance. However, t… (see more)he practical utility of such estimates is limited by how accurately they quantify predictive uncertainty. In this work, we address the problem of estimating the joint predictive distribution of high-dimensional multivariate time series. We propose a versatile method, based on the transformer architecture, that estimates joint distributions using an attention-based decoder that provably learns to mimic the properties of non-parametric copulas. The resulting model has several desirable properties: it can scale to hundreds of time series, supports both forecasting and interpolation, can handle unaligned and non-uniformly sampled data, and can seamlessly adapt to missing data during training. We demonstrate these properties empirically and show that our model produces state-of-the-art predictions on multiple real-world datasets.
2022-02-07
ArXiv (preprint)
arxiv.org
arxiv.org
ASHA: Assistive Teleoperation via Human-in-the-Loop Reinforcement Learning
Sean Andrew Chen
Jensen Gao
Siddharth Reddy
Glen Berseth
Anca Dragan
Sergey Levine
Building assistive interfaces for controlling robots through arbitrary, high-dimensional, noisy inputs (e.g., webcam images of eye gaze) can… (see more) be challenging, especially when it involves inferring the user's desired action in the absence of a natural ‘default’ interface. Reinforcement learning from online user feedback on the system's performance presents a natural solution to this problem, and enables the interface to adapt to individual users. However, this approach tends to require a large amount of human-in-the-loop training data, especially when feedback is sparse. We propose a hierarchical solution that learns efficiently from sparse user feedback: we use offline pre-training to acquire a latent embedding space of useful, high-level robot behaviors, which, in turn, enables the system to focus on using online user feedback to learn a mapping from user inputs to desired high-level behaviors. The key insight is that access to a pre-trained policy enables the system to learn more from sparse rewards than a naïve RL algorithm: using the pre-trained policy, the system can make use of successful task executions to relabel, in hindsight, what the user actually meant to do during unsuccessful executions. We evaluate our method primarily through a user study with 12 participants who perform tasks in three simulated robotic manipulation domains using a webcam and their eye gaze: flipping light switches, opening a shelf door to reach objects inside, and rotating a valve. The results show that our method successfully learns to map 128-dimensional gaze features to 7-dimensional joint torques from sparse rewards in under 10 minutes of online training, and seamlessly helps users who employ different gaze strategies, while adapting to distributional shift in webcam inputs, tasks, and environments
2022-02-05
ArXiv (preprint)
doi.org
arxiv.org
TIML: Task-Informed Meta-Learning for Agriculture
Gabriel Tseng
Hannah Kerner
David Rolnick
Labeled datasets for agriculture are extremely spatially imbalanced. When developing algorithms for data-sparse regions, a natural approach … (see more)is to use transfer learning from data-rich regions. While standard transfer learning approaches typically leverage only direct inputs and outputs, geospatial imagery and agricultural data are rich in metadata that can inform transfer learning algorithms, such as the spatial coordinates of data-points or the class of task being learned. We build on previous work exploring the use of meta-learning for agricultural contexts in data-sparse regions and introduce task-informed meta-learning (TIML), an augmentation to model-agnostic meta-learning which takes advantage of task-specific metadata. We apply TIML to crop type classification and yield estimation, and find that TIML significantly improves performance compared to a range of benchmarks in both contexts, across a diversity of model architectures. While we focus on tasks from agriculture, TIML could offer benefits to any meta-learning setup with task-specific metadata, such as classification of geo-tagged images and species distribution modelling.
2022-02-04
ArXiv (preprint)
arxiv.org
arxiv.org
Quantum-Inspired Interpertable AI-Empowered Decision Support System for Detection of Early-Stage Rheumatoid Arthritis in Primary Care Using Scarce Dataset
Samira Abbasgholizadeh-Rahimi
Mojtaba Kolahdoozi
Arka Mitra
Jose L Salmeron
Amir-Mohammad Navali
Alireza Sadeghpour
Amir Mir Mir Mohammadi
2022-02-03
Mathematics (published)
doi.org
Active Learning for Capturing Human Decision Policies in a Data Frugal Context
Loïc Grossetête
Alexandre Marois
Bénédicte Chatelais
Christian Gagné
Daniel Lafond
2022-02-02
Machine Learning, Optimization, and Data Science (published)
doi.org