Publications

Deep learning, reinforcement learning, and world models
Yu Matsuo
Yann Lecun
Maneesh Sahani
Doina Precup
David Silver
Masashi Sugiyama
Eiji Uchibe
J. Morimoto
2022-03-31
Neural Networks (publié)
doi.org
DyAdvDefender: An instance-based online machine learning model for perturbation-trial-based black-box adversarial defense
Miles Q. Li
Benjamin C. M. Fung
Philippe Charland
2022-03-31
Information Sciences (publié)
doi.org
Multivariate, Transgenerational Associations of the COVID-19 Pandemic Across Minoritized and Marginalized Communities.
Sarah W. Yip
Ayana Jordan
Robert J. Kohler
Avram J. Holmes
Danilo Bzdok
Importance The experienced consequences of the COVID-19 pandemic have diverged across individuals, families, and communities, resulting in i… (voir plus)nequity within a host of factors. There is a gap of quantitative evidence about the transgenerational impacts of these experiences and factors. Objective To identify baseline predictors of COVID-19 experiences, as defined by child and parent report, using a multivariate pattern-learning framework from the Adolescent Brain and Cognitive Development (ABCD) cohort. Design, Setting, and Participants ABCD is an ongoing prospective longitudinal study of child and adolescent development in the United States including 11 875 youths, enrolled at age 9 to 10 years. Using nationally collected longitudinal profiling data from 9267 families, a multivariate pattern-learning strategy was developed to identify factor combinations associated with transgenerational costs of the ongoing COVID-19 pandemic. ABCD data (release 3.0) collected from 2016 to 2020 and released between 2019 and 2021 were analyzed in combination with ABCD COVID-19 rapid response data from the first 3 collection points (May-August 2020). Exposures Social distancing and other response measures imposed by COVID-19, including school closures and shutdown of many childhood recreational activities. Main Outcomes and Measures Mid-COVID-19 experiences as defined by the ABCD's parent and child COVID-19 assessments. Results Deep profiles from 9267 youth (5681 female [47.8%]; mean [SD] age, 119.0 [7.5] months) and their caregivers were quantitatively examined. Enabled by a pattern-learning analysis, social determinants of inequity, including family structure, socioeconomic status, and the experience of racism, were found to be primarily associated with transgenerational impacts of COVID-19, above and beyond other candidate predictors such as preexisting medical or psychiatric conditions. Pooling information across more than 17 000 baseline pre-COVID-19 family indicators and more than 280 measures of day-to-day COVID-19 experiences, non-White (ie, families who reported being Asian, Black, Hispanic, other, or a combination of those choices) and/or Spanish-speaking families were found to have decreased resources (mode 1, canonical vector weight [CVW] = 0.19; rank 5 of 281), escalated likelihoods of financial worry (mode 1, CVW = -0.20; rank 4), and food insecurity (mode 1, CVW = 0.21; rank 2), yet were more likely to have parent-child discussions regarding COVID-19-associated health and prevention issues, such as handwashing (mode 1, CVW = 0.14; rank 9), conserving food or other items (mode 1, CVW = 0.21; rank 1), protecting elderly individuals (mode 1, CVW = 0.11; rank 21), and isolating from others (mode 1, CVW = 0.11; rank 23). In contrast, White families (mode 1, CVW = -0.07; rank 3), those with higher pre-COVID-19 income (mode 1, CVW = -0.07; rank 5), and presence of a parent with a postgraduate degree (mode 1, CVW = -0.06; rank 14) experienced reduced COVID-19-associated impact. In turn, children from families experiencing reduced COVID-19 impacts reported longer nighttime sleep durations (mode 1, CVW = 0.13; rank 14), less difficulties with remote learning (mode 2, CVW = 0.14; rank 7), and decreased worry about the impact of COVID-19 on their family's financial stability (mode 1, CVW = 0.134; rank 13). Conclusions and Relevance The findings of this study indicate that community-level, transgenerational intervention strategies may be needed to combat the disproportionate burden of pandemics on minoritized and marginalized racial and ethnic populations.
2022-03-31
JAMA Psychiatry (publié)
doi.org
Pattern learning reveals brain asymmetry to be linked to socioeconomic status
Timm B. Poeppl
Emile Dimas
Katrin Sakreida
Julius M. Kernbach
Ross D. Markello
Oliver Schöffski
Alain Dagher
Philipp Koellinger
Gideon Nave
Martha J. Farah
Bratislav Misic
Danilo Bzdok
Socioeconomic status (SES) anchors individuals in their social network layers. Our embedding in the societal fabric resonates with habitus, … (voir plus)world view, opportunity, and health disparity. It remains obscure how distinct facets of SES are reflected in the architecture of the central nervous system. Here, we capitalized on multivariate multi-output learning algorithms to explore possible imprints of SES in gray and white matter structure in the wider population (n ≈ 10,000 UK Biobank participants). Individuals with higher SES, compared with those with lower SES, showed a pattern of increased region volumes in the left brain and decreased region volumes in the right brain. The analogous lateralization pattern emerged for the fiber structure of anatomical white matter tracts. Our multimodal findings suggest hemispheric asymmetry as an SES-related brain signature, which was consistent across six different indicators of SES: degree, education, income, job, neighborhood and vehicle count. Hence, hemispheric specialization may have evolved in human primates in a way that reveals crucial links to SES.
2022-03-31
Cerebral Cortex Communications (publié)
doi.org
Population heterogeneity in clinical cohorts affects the predictive accuracy of brain imaging
Oualid Benkarim
Casey Paquola
Bo-yong Park
Valeria Kebets
Seok-Jun Hong
Reinder Vos de Wael
Shaoshi Zhang
B. T. Thomas Yeo
Michael Eickenberg
Tian Ge
Jean-Baptiste Poline
Boris C. Bernhardt
Danilo Bzdok
Brain imaging research enjoys increasing adoption of supervised machine learning for single-participant disease classification. Yet, the suc… (voir plus)cess of these algorithms likely depends on population diversity, including demographic differences and other factors that may be outside of primary scientific interest. Here, we capitalize on propensity scores as a composite confound index to quantify diversity due to major sources of population variation. We delineate the impact of population heterogeneity on the predictive accuracy and pattern stability in 2 separate clinical cohorts: the Autism Brain Imaging Data Exchange (ABIDE, n = 297) and the Healthy Brain Network (HBN, n = 551). Across various analysis scenarios, our results uncover the extent to which cross-validated prediction performances are interlocked with diversity. The instability of extracted brain patterns attributable to diversity is located preferentially in regions part of the default mode network. Collectively, our findings highlight the limitations of prevailing deconfounding practices in mitigating the full consequences of population diversity.
2022-03-31
PLoS Biology (publié)
doi.org
Planning as Inference in Epidemiological Dynamics Models
Frank Wood
Andrew Warrington
Saeid Naderiparizi
Christian Weilbach
Vaden Masrani
William Harvey
Adam Ścibior
Boyan Beronov
Ali Nasseri
In this work we demonstrate how to automate parts of the infectious disease-control policy-making process via performing inference in existi… (voir plus)ng epidemiological models. The kind of inference tasks undertaken include computing the posterior distribution over controllable, via direct policy-making choices, simulation model parameters that give rise to acceptable disease progression outcomes. Among other things, we illustrate the use of a probabilistic programming language that automates inference in existing simulators. Neither the full capabilities of this tool for automating inference nor its utility for planning is widely disseminated at the current time. Timely gains in understanding about how such simulation-based models and inference automation tools applied in support of policymaking could lead to less economically damaging policy prescriptions, particularly during the current COVID-19 pandemic.
2022-03-30
Frontiers in Artificial Intelligence (publié)
doi.org
arxiv.org
From Points to Functions: Infinite-dimensional Representations in Diffusion Models
Sarthak Mittal
Guillaume Lajoie
Stefan Bauer
Arash Mehrjou
Diffusion-based generative models learn to iteratively transfer unstructured noise to a complex target distribution as opposed to Generative… (voir plus) Adversarial Networks (GANs) or the decoder of Variational Autoencoders (VAEs) which produce samples from the target distribution in a single step. Thus, in diffusion models every sample is naturally connected to a random trajectory which is a solution to a learned stochastic differential equation (SDE). Generative models are only concerned with the final state of this trajectory that delivers samples from the desired distribution. Abstreiter et. al showed that these stochastic trajectories can be seen as continuous filters that wash out information along the way. Consequently, it is reasonable to ask if there is an intermediate time step at which the preserved information is optimal for a given downstream task. In this work, we show that a combination of information content from different time steps gives a strictly better representation for the downstream task. We introduce an attention and recurrence based modules that ``learn to mix'' information content of various time-steps such that the resultant representation leads to superior performance in downstream tasks.
2022-03-27
ICLR.cc/2022/Workshop/DGM4HSD (poster)
doi.org
openreview.net
Reproducibility and Evolution of Diffusion Mri Measurements Within the Cervical Spinal Cord in Multiple Sclerosis
Haykel Snoussi
Emmanuel Caruyer
Benoit Combes
Olivier Commowick
Élise Bannier
Anne Kerbrat
Julien Cohen-Adad
Christian Barillot
In Multiple Sclerosis (MS), there is a large discrepancy between the clinical observations and how the pathology is exhibited on brain image… (voir plus)s, this is known as the clinical-radiological paradox. One of the hypotheses is that the clinical deficit may be more related to the spinal cord damage than the number or location of lesions in the brain. Therefore, investigating how the spinal cord is damaged becomes an acute challenge to better understand and overcome this paradox. Diffusion MRI is known to provide quantitative figures of neuronal degeneration and axonal loss, in the brain as well as in the spinal cord. In this paper, we propose to investigate how diffusion MRI metrics vary in the different cervical regions with the progression of the disease. We first study the reproducibility of diffusion MRI on healthy volunteers with a test-retest procedure using both standard diffusion tensor imaging (DTI) and multi-compartment Ball-and-Stick models. Then, based on the test re-test quantitative calibration, we provide quantitative figures of pathology evolution between M0 and M12 in the cervical spine on a set of 31 MS patients, exhibiting how the pathology damage spans in the cervical spinal cord.
2022-03-27
2022 IEEE 19th International Symposium on Biomedical Imaging (ISBI) (publié)
doi.org
arxiv.org
Structured Multi-task Learning for Molecular Property Prediction
Shengchao Liu
Meng Qu
Zuobai Zhang
Huiyu Cai
Jian Tang
Multi-task learning for molecular property prediction is becoming increasingly important in drug discovery. However, in contrast to other do… (voir plus)mains, the performance of multi-task learning in drug discovery is still not satisfying as the number of labeled data for each task is too limited, which calls for additional data to complement the data scarcity. In this paper, we study multi-task learning for molecular property prediction in a novel setting, where a relation graph between tasks is available. We first construct a dataset (ChEMBL-STRING) including around 400 tasks as well as a task relation graph. Then to better utilize such relation graph, we propose a method called SGNN-EBM to systematically investigate the structured task modeling from two perspectives. (1) In the \emph{latent} space, we model the task representations by applying a state graph neural network (SGNN) on the relation graph. (2) In the \emph{output} space, we employ structured prediction with the energy-based model (EBM), which can be efficiently trained through noise-contrastive estimation (NCE) approach. Empirical results justify the effectiveness of SGNN-EBM. Code is available on https://github.com/chao1224/SGNN-EBM.
2022-03-27
International Conference on Artificial Intelligence and Statistics (Accept (Poster))
doi.org
proceedings.mlr.press
On the Convergence of Continuous Constrained Optimization for Structure Learning
Ignavier Ng
Sébastien Lachapelle
Nan Rosemary Ke
Simon Lacoste-Julien
Kun Zhang
Recently, structure learning of directed acyclic graphs (DAGs) has been formulated as a continuous optimization problem by leveraging an alg… (voir plus)ebraic characterization of acyclicity. The constrained problem is solved using the augmented Lagrangian method (ALM) which is often preferred to the quadratic penalty method (QPM) by virtue of its standard convergence result that does not require the penalty coefficient to go to infinity, hence avoiding ill-conditioning. However, the convergence properties of these methods for structure learning, including whether they are guaranteed to return a DAG solution, remain unclear, which might limit their practical applications. In this work, we examine the convergence of ALM and QPM for structure learning in the linear, nonlinear, and confounded cases. We show that the standard convergence result of ALM does not hold in these settings, and demonstrate empirically that its behavior is akin to that of the QPM which is prone to ill-conditioning. We further establish the convergence guarantee of QPM to a DAG solution, under mild conditions. Lastly, we connect our theoretical results with existing approaches to help resolve the convergence issue, and verify our findings in light of an empirical comparison of them.
2022-03-27
International Conference on Artificial Intelligence and Statistics (Accept (Poster))
doi.org
proceedings.mlr.press
On the Convergence of Stochastic Extragradient for Bilinear Games using Restarted Iteration Averaging
Chris Junchi Li
Yi Ma
Yaodong Yu
Nicolas Loizou
Gauthier Gidel
Nicolas Le Roux
Michael I. Jordan
We study the stochastic bilinear minimax optimization problem, presenting an analysis of the same-sample Stochastic ExtraGradient (SEG) meth… (voir plus)od with constant step size, and presenting variations of the method that yield favorable convergence. In sharp contrasts with the basic SEG method whose last iterate only contracts to a fixed neighborhood of the Nash equilibrium, SEG augmented with iteration averaging provably converges to the Nash equilibrium under the same standard settings, and such a rate is further improved by incorporating a scheduled restarting procedure. In the interpolation setting where noise vanishes at the Nash equilibrium, we achieve an optimal convergence rate up to tight constants. We present numerical experiments that validate our theoretical findings and demonstrate the effectiveness of the SEG method when equipped with iteration averaging and restarting.
2022-03-27
International Conference on Artificial Intelligence and Statistics (Accept (Poster))
proceedings.mlr.press
proceedings.mlr.press
Inductive Biases for Relational Tasks
Giancarlo Kerg
Sarthak Mittal
David Rolnick
Yoshua Bengio
Blake Aaron Richards
Guillaume Lajoie
Current deep learning approaches have shown good in-distribution performance but struggle in out-of-distribution settings. This is especiall… (voir plus)y true in the case of tasks involving abstract relations like recognizing rules in sequences, as required in many intelligence tests. In contrast, our brains are remarkably flexible at such tasks, an attribute that is likely linked to anatomical constraints on computations. Inspired by this, recent work has explored how enforcing that relational representations remain distinct from sensory representations can help artificial systems. Building on this work, we further explore and formalize the advantages afforded by ``partitioned'' representations of relations and sensory details. We investigate inductive biases that ensure abstract relations are learned and represented distinctly from sensory data across several neural network architectures and show that they outperform existing architectures on out-of-distribution generalization for various relational tasks. These results show that partitioning relational representations from other information streams may be a simple way to augment existing network architectures' robustness when performing relational computations.
2022-03-24
ICLR.cc/2022/Workshop/OSC (poster)
openreview.net
openreview.net