Portrait de Sumana Basu

Sumana Basu

Collaborateur·rice alumni - McGill
Superviseur⋅e principal⋅e
Doina Precup
Co-supervisor
Adriana Romero Soriano
Sujets de recherche
Apprentissage par renforcement
Site web
Google Scholar
GitHub

Publications

Reward the Reward Designer: Making Reinforcement Learning Useful for Clinical Decision Making
Sumana Basu
Adriana Romero
Doina Precup
2025-09-21
NeurIPS.cc/2025/Workshop/WiML (publié)
openreview.net
openreview.net
On the Challenges of using Reinforcement Learning in Precision Drug Dosing: Delay and Prolongedness of Action Effects
Sumana Basu
M. Legault
Adriana Romero
Doina Precup
Drug dosing is an important application of AI, which can be formulated as a Reinforcement Learning (RL) problem. In this paper, we identify … (voir plus)two major challenges of using RL for drug dosing: delayed and prolonged effects of administering medications, which break the Markov assumption of the RL framework. We focus on prolongedness and define PAE-POMDP (Prolonged Action Effect-Partially Observable Markov Decision Process), a subclass of POMDPs in which the Markov assumption does not hold specifically due to prolonged effects of actions. Motivated by the pharmacology literature, we propose a simple and effective approach to converting drug dosing PAE-POMDPs into MDPs, enabling the use of the existing RL algorithms to solve such problems. We validate the proposed approach on a toy task, and a challenging glucose control task, for which we devise a clinically-inspired reward function. Our results demonstrate that: (1) the proposed method to restore the Markov assumption leads to significant improvements over a vanilla baseline; (2) the approach is competitive with recurrent policies which may inherently capture the prolonged affect of actions; (3) it is remarkably more time and memory efficient than the recurrent baseline and hence more suitable for real-time dosing control systems; and (4) it exhibits favourable qualitative behavior in our policy analysis.
2023-01-01
ArXiv (prépublication)
doi.org
arxiv.org
Towards Safe Mechanical Ventilation Treatment Using Deep Offline Reinforcement Learning
Flemming Kondrup
Thomas Jiralerspong
Elaine Lau
Nathan de Lara
Jacob Shkrob
My Duc Tran
Doina Precup
Sumana Basu
Mechanical ventilation is a key form of life support for patients with pulmonary impairment. Healthcare workers are required to continuously… (voir plus) adjust ventilator settings for each patient, a challenging and time consuming task. Hence, it would be beneficial to develop an automated decision support tool to optimize ventilation treatment. We present DeepVent, a Conservative Q-Learning (CQL) based offline Deep Reinforcement Learning (DRL) agent that learns to predict the optimal ventilator parameters for a patient to promote 90 day survival. We design a clinically relevant intermediate reward that encourages continuous improvement of the patient vitals as well as addresses the challenge of sparse reward in RL. We find that DeepVent recommends ventilation parameters within safe ranges, as outlined in recent clinical trials. The CQL algorithm offers additional safety by mitigating the overestimation of the value estimates of out-of-distribution states/actions. We evaluate our agent using Fitted Q Evaluation (FQE) and demonstrate that it outperforms physicians from the MIMIC-III dataset.
2022-10-04
ArXiv (prépublication)
doi.org
arxiv.org
Option-Critic in Cooperative Multi-Agent Systems
Jhelum Chakravorty
Nadeem Ward
Julien Roy
Maxime Chevalier-Boisvert
Sumana Basu
Andrei Lupu
Doina Precup
In this paper, we investigate learning temporal abstractions in cooperative multi-agent systems, using the options framework (Sutton et al, … (voir plus)1999). First, we address the planning problem for the decentralized POMDP represented by the multi-agent system, by introducing a \emph{common information approach}. We use the notion of \emph{common beliefs} and broadcasting to solve an equivalent centralized POMDP problem. Then, we propose the Distributed Option Critic (DOC) algorithm, which uses centralized option evaluation and decentralized intra-option improvement. We theoretically analyze the asymptotic convergence of DOC and build a new multi-agent environment to demonstrate its validity. Our experiments empirically show that DOC performs competitively against baselines and scales with the number of agents.
2020-05-04
International Joint Conference on Autonomous Agents and Multiagent Systems (publié)
doi.org
arxiv.org
Early Prediction of Alzheimer's Disease Progression Using Variational Autoencoders
Sumana Basu
Konrad Wagstyl
Azar Zandifar
D. Collins
Adriana Romero
Doina Precup
2019-10-09
Lecture Notes in Computer Science (publié)
doi.org
Analyzing Alzheimer’s Disease Progression from Sequential Magnetic Resonance Imaging Scans Using Deep 3D Convolutional Neural Networks
Sumana Basu
Konrad Wagstyl
Azar Zandifar
Louis Collins
Adriana Romero
Doina Precup
Alzheimer’s is a progressive, neurodegenerative disease, that causes irreversible damage to the brain tissue. It impairs the ability to fo… (voir plus)rm and retrieve memory, and eventually disrupts the natural flow of life, by affecting the ability to carry out even day to day activities. The disease is typically diagnosed from the symptoms (Mini Mental State Examination, [8]), such as decline in cognitive abilities, visual and/or speech impairment, loss of memory, rather than the structural changes in the brain (biomarker) that causes it. But the pathological changes in the brain start decades before the manifestation of the symptoms [7]. Magnetic Resonance Imaging (MRI) is capable of capturing the complex changes in the brain, even if it is difficult for humans to extract those features from the low contrast, multi-dimensional MRIs [1]. There is a considerable amount of work on analyzing Alzheimer’s disease. However, the vast majority intends to predict the state of the disease at the current time step.
2017-12-31
(publié)
www.semanticscholar.org