Publications

Switching between tasks can cause AI to lose the ability to learn
Clare Lyle
Razvan Pascanu
2024-07-31
Nature (publié)
doi.org
Understanding Decision-Time vs. Background Planning in Model-Based Reinforcement Learning
Safa Alver
Doina Precup
In model-based reinforcement learning, an agent can leverage a learned model to improve its way of behaving in different ways. Two prevalent… (voir plus) approaches are decision-time planning and background planning. In this study, we are interested in understanding under what conditions and in which settings one of these two planning styles will perform better than the other in domains that require fast responses. After viewing them through the lens of dynamic programming, we first consider the classical instantiations of these planning styles and provide theoretical results and hypotheses on which one will perform better in the pure planning, planning&learning, and transfer learning settings. We then consider the modern instantiations of these planning styles and provide hypotheses on which one will perform better in the last two of the considered settings. Lastly, we perform several illustrative experiments to empirically validate both our theoretical results and hypotheses. Overall, our findings suggest that even though decision-time planning does not perform as well as background planning in their classical instantiations, in their modern instantiations, it can perform on par or better than background planning in both the planning&learning and transfer learning settings.
2024-07-31
EWRL/2024/Workshop (accepté)
doi.org
openreview.net
scHiCyclePred: a deep learning framework for predicting cell cycle phases from single-cell Hi-C data using multi-scale interaction information
Yingfu Wu
Zhenqi Shi
Xiangfei Zhou
Pengyu Zhang
Xiuhui Yang
Jun Ding
Hao Wu
2024-07-30
Communications Biology (publié)
doi.org
Diversity-aware Population Models: Quantifying Associations between Socio-Spatial Factors and Cognitive Development in the ABCD Cohort
Nicole Osayande
Justin Marotta
Shambhavi Aggarwal
Jakub Kopal
Avram Holmes
Sarah W. Yip
Danilo Bzdok
Population-level analyses are inherently complex due to a myriad of latent confounding effects that underlie the interdisciplinary topics of… (voir plus) research interest. Despite the mounting demand for generative population models, the limited generalizability to underrepresented groups hinders their widespread adoption in downstream applications. Interpretability and reliability are essential for clinicians and policymakers, while accuracy and precision are prioritized from an engineering standpoint. Thus, in domains such as population neuroscience, the challenge lies in determining a suitable approach to model population data effectively. Notably, the traditional strata-agnostic nature of existing methods in this field reveals a pertinent gap in quantitative techniques that directly capture major sources of population stratification. The emergence of population-scale cohorts, like the Adolescent Brain Cognitive DevelopmentSM (ABCD) Study, provides unparalleled opportunities to explore and characterize neurobehavioral and sociodemographic relationships comprehensively. We propose diversity-aware population modeling, a framework poised to standardize systematic incorporation of diverse attributes, structured with respect to intrinsic population stratification to obtain holistic insights. Here, we leverage Bayesian multilevel regression and poststratification, to elucidate inter-individual differences in the relationships between socioeconomic status (SES) and cognitive development. We constructed 14 varying-intercepts and varying-slopes models to investigate 3 cognitive phenotypes and 5 sociodemographic variables (SDV), across 17 US states and 5 race subgroups. SDVs exhibited systemic socio-spatial effects that served as fundamental drivers of variation in cognitive outcomes. Low SES was disproportionately associated with cognitive development among Black and Hispanic children, while high SES was a robust predictor of cognitive development only among White and Asian children, consistent with the minorities’ diminished returns (MDRs) theory. Notably, adversity-susceptible subgroups demonstrated an expressive association with fluid cognition compared to crystallized cognition. Poststratification proved effective in correcting group attribution biases, particularly in Pennsylvania, highlighting sampling discrepancies in US states with the highest percentage of marginalized participants in the ABCD Study©. Our collective analyses underscore the inextricable link between race and geographic location within the US. We emphasize the importance of diversity-aware population models that consider the intersectional composition of society to derive precise and interpretable insights across applicable domains.
2024-07-29
Research Square (publié)
doi.org
Strong Gravitational Lensing as a Probe of Dark Matter
S. Vegetti
S. Birrer
G. Despali
C.D. Fassnacht
D. Gilman
Y. Hezaveh
L.
L. Perreault Levasseur
J.P. McKean
D.M. Powell
C.M. O'Riordan
G.
G. Vernardos
Dark matter structures within strong gravitational lens galaxies and along their line of sight leave a gravitational imprint on the multiple… (voir plus) images of lensed sources. Strong gravitational lensing provides, therefore, a key test of different dark matter models in a way that is independent of the baryonic content of matter structures on subgalactic scales. In this chapter, we describe how galaxy-scale strong gravitational lensing observations are sensitive to the physical nature of dark matter. We provide a historical perspective of the field, and review its current status. We discuss the challenges and advances in terms of data, treatment of systematic errors and theoretical predictions, that will enable one to deliver a stringent and robust test of different dark matter models in the near future. With the advent of the next generation of sky surveys, the number of known strong gravitational lens systems is expected to increase by several orders of magnitude. Coupled with high-resolution follow-up observations, these data will provide a key opportunity to constrain the properties of dark matter with strong gravitational lensing.
2024-07-29
Space Science Reviews (publié)
doi.org
arxiv.org
AAPM task group report 288: Recommendations for guiding radiotherapy event narratives
Bruce Thomadsen
Ajay Kapur
Bette Blankenship
Barrett Caldwell
Lindsey Claps
Joanne Cunningham
Jennifer Elee
Suzanne Evans
Eric Ford
Debbie Gilley
Sandra Hayden
Kathleen Hintenlang
Rishabh Kapoor
J. Kildea
Linda Kroger
Ksenija Kujundzic
Qing Liang
Sasa Mutic
Anita O'Donovan
Michael O'Hara … (voir 6 de plus)
Zoubir Ouhib
Jatinder Palta
Todd Pawlicki
William Salter
Stacey Schmidt
Sugata Tripathi
2024-07-28
Medical Physics (Lancaster) (publié)
doi.org
scCross: a deep generative model for unifying single-cell multi-omics with seamless integration, cross-modal generation, and in silico exploration
Xiuhui Yang
Koren K. Mann
Hao Wu
Jun Ding
Single-cell multi-omics data reveal complex cellular states, providing significant insights into cellular dynamics and disease. Yet, integra… (voir plus)tion of multi-omics data presents challenges. Some modalities have not reached the robustness or clarity of established transcriptomics. Coupled with data scarcity for less established modalities and integration intricacies, these challenges limit our ability to maximize single-cell omics benefits. We introduce scCross, a tool leveraging variational autoencoders, generative adversarial networks, and the mutual nearest neighbors (MNN) technique for modality alignment. By enabling single-cell cross-modal data generation, multi-omics data simulation, and in silico cellular perturbations, scCross enhances the utility of single-cell multi-omics studies. The online version contains supplementary material available at 10.1186/s13059-024-03338-z.
2024-07-28
Genome Biology (publié)
doi.org
The report of AAPM task group 288: Recommendations for guiding radiotherapy event narratives.
Bruce Thomadsen
Ajay Kapur
Bette Blankenship
Barrett Caldwell
Lindsey Claps
Joanne Cunningham
Jennifer Elee
Suzanne Evans
Eric Ford
Debbie Gilley
Sandra Hayden
Kathleen Hintenlang
Rishabh Kapoor
J. Kildea
Linda Kroger
Ksenija Kujundzic
Qing Liang
Sasa Mutic
Anita O'Donovan
Michael O'Hara … (voir 6 de plus)
Zoubir Ouhib
Jatinder Palta
Todd Pawlicki
William Salter
Stacey Schmidt
Sugata Tripathi
Incident reporting and learning systems provide an opportunity to identify systemic vulnerabilities that contribute to incidents and potenti… (voir plus)ally degrade quality. The narrative of an incident is intended to provide a clear, easy to understand description of an incident. Unclear, incomplete or poorly organized narratives compromise the ability to learn from them. This report provides guidance for drafting effective narratives, with particular attention to the use of narratives in incident reporting and learning systems (IRLS). Examples are given that compare effective and less than effective narratives. This report is mostly directed to organizations that maintain IRLS, but also may be helpful for individuals who desire to write a useful narrative for entry into such a system. Recommendations include the following: (1) Systems should allow a one- or two-sentence, free-text synopsis of an incident without guessing at causes; (2) Information included should form a sequence of events with chronology; and (3) Reporting and learning systems should consider using the headings suggested to guide the reporter through the narrative: (a) incident occurrences and actions by role; (b) prior circumstances and actions; (c) method by which the incident was identified; (d) equipment related details if relevant; (e) recovery actions by role; (f) relevant time span between responses; (g) and how individuals affected during or immediately after incident. When possible and appropriate, supplementary information including relevant data elements should be included using numerical scales or drop-down choices outside of the narrative. Information that should not be included in the narrative includes: (a) patient health information (PHI); (b) conjecture or blame; (c) jargon abbreviations or details without specifying their significance; (d) causal analysis.
2024-07-28
Medical Physics (Lancaster) (publié)
doi.org
Empowering Clinicians with Medical Decision Transformers: A Framework for Sepsis Treatment
Aamer Abdul Rahman
Pranav Agarwal
Rita Noumeir
Philippe Jouvet
Vincent Michalski
S Ebrahimi Kahou
Offline reinforcement learning has shown promise for solving tasks in safety-critical settings, such as clinical decision support. Its appli… (voir plus)cation, however, has been limited by the lack of interpretability and interactivity for clinicians. To address these challenges, we propose the medical decision transformer (MeDT), a novel and versatile framework based on the goal-conditioned reinforcement learning paradigm for sepsis treatment recommendation. MeDT uses the decision transformer architecture to learn a policy for drug dosage recommendation. During offline training, MeDT utilizes collected treatment trajectories to predict administered treatments for each time step, incorporating known treatment outcomes, target acuity scores, past treatment decisions, and current and past medical states. This analysis enables MeDT to capture complex dependencies among a patient's medical history, treatment decisions, outcomes, and short-term effects on stability. Our proposed conditioning uses acuity scores to address sparse reward issues and to facilitate clinician-model interactions, enhancing decision-making. Following training, MeDT can generate tailored treatment recommendations by conditioning on the desired positive outcome (survival) and user-specified short-term stability improvements. We carry out rigorous experiments on data from the MIMIC-III dataset and use off-policy evaluation to demonstrate that MeDT recommends interventions that outperform or are competitive with existing offline reinforcement learning methods while enabling a more interpretable, personalized and clinician-directed approach.
2024-07-27
ArXiv (prépublication)
doi.org
arxiv.org
On the benefits of pixel-based hierarchical policies for task generalization
T. Cristea-Platon
Bogdan Mazoure
Josh Susskind
Walter Talbott
Reinforcement learning practitioners often avoid hierarchical policies, especially in image-based observation spaces. Typically, the single-… (voir plus)task performance improvement over flat-policy counterparts does not justify the additional complexity associated with implementing a hierarchy. However, by introducing multiple decision-making levels, hierarchical policies can compose lower-level policies to more effectively generalize between tasks, highlighting the need for multi-task evaluations. We analyze the benefits of hierarchy through simulated multi-task robotic control experiments from pixels. Our results show that hierarchical policies trained with task conditioning can (1) increase performance on training tasks, (2) lead to improved reward and state-space generalizations in similar tasks, and (3) decrease the complexity of fine tuning required to solve novel tasks. Thus, we believe that hierarchical policies should be considered when building reinforcement learning architectures capable of generalizing between tasks.
2024-07-26
ArXiv (prépublication)
doi.org
arxiv.org
Canada's Provincial Covid-19 Pandemic Modelling Efforts: A Review of Mathematical Models and Their Impacts on the Responses
Yiqing Xia
Jorge Luis Flores Anato
Caroline Colijin
Naveed Janjua
Michael Otterstatter
Mike Irvine
Tyler Williamson
Marie B. Varughese
Michael Li
Nathaniel Osgood
David J. D. Earn
Beate Sander
Lauren E. Cipriano
Kumar Murty
Fanyu Xiu
Arnaud Godin
David L Buckeridge
Amy Hurford
Sharmistha Mishra
Mathieu Maheu-Giroux
2024-07-24
Canadian Journal of Public Health (publié)
doi.org
Enhancing Agent Learning through World Dynamics Modeling
Zhiyuan Sun
Haochen Shi
Marc-Alexandre Côté
Glen Berseth
Xingdi Yuan
Bang Liu
Large language models (LLMs) have been increasingly applied to tasks in language understanding and interactive decision-making, with their i… (voir plus)mpressive performance largely attributed to the extensive domain knowledge embedded within them. However, the depth and breadth of this knowledge can vary across domains. Many existing approaches assume that LLMs possess a comprehensive understanding of their environment, often overlooking potential gaps in their grasp of actual world dynamics. To address this, we introduce Discover, Verify, and Evolve (DiVE), a framework that discovers world dynamics from a small number of demonstrations, verifies the accuracy of these dynamics, and evolves new, advanced dynamics tailored to the current situation. Through extensive evaluations, we assess the impact of each component on performance and compare the dynamics generated by DiVE to human-annotated dynamics. Our results show that LLMs guided by DiVE make more informed decisions, achieving rewards comparable to human players in the Crafter environment and surpassing methods that require prior task-specific training in the MiniHack environment.
2024-07-24
arXiv (Cornell University) (prépublication)
doi.org
arxiv.org