Tal Arbel

Google Scholar

Gian Favero

Maîtrise recherche - McGill University

gian.favero@mila.quebec

Site web

mohamed.mohamed@mila.quebec

Joshua Durso-finley

Doctorat - McGill University

dursofij@mila.quebec

Mohamed Mohamed

Maîtrise recherche - McGill University

Nima Fathi

Maîtrise recherche - McGill University

nima.fathi@mila.quebec

Site web

parham.saremi@mila.quebec

Parham Saremi

Maîtrise recherche - McGill University

zahra.tehraninasab@mila.quebec

Xing Shen

Maîtrise recherche - McGill University

xing.shen@mila.quebec

Maîtrise recherche - McGill University

Publications

HyperFusion: A Hypernetwork Approach to Multimodal Integration of Tabular and Medical Imaging Data for Predictive Modeling

Daniel Duenias

Brennan Nichyporuk

Tammy Riklin-Raviv

The integration of diverse clinical modalities such as medical imaging and the tabular data obtained by the patients' Electronic Health Reco… (voir plus)rds (EHRs) is a crucial aspect of modern healthcare. The integrative analysis of multiple sources can provide a comprehensive understanding of a patient's condition and can enhance diagnoses and treatment decisions. Deep Neural Networks (DNNs) consistently showcase outstanding performance in a wide range of multimodal tasks in the medical domain. However, the complex endeavor of effectively merging medical imaging with clinical, demographic and genetic information represented as numerical tabular data remains a highly active and ongoing research pursuit. We present a novel framework based on hypernetworks to fuse clinical imaging and tabular data by conditioning the image processing on the EHR's values and measurements. This approach aims to leverage the complementary information present in these modalities to enhance the accuracy of various medical applications. We demonstrate the strength and the generality of our method on two different brain Magnetic Resonance Imaging (MRI) analysis tasks, namely, brain age prediction conditioned by subject's sex, and multiclass Alzheimer's Disease (AD) classification conditioned by tabular data. We show that our framework outperforms both single-modality models and state-of-the-art MRI-tabular data fusion methods. The code, enclosed to this manuscript will be made publicly available.

2024-03-20

ArXiv (prépublication)

DeCoDEx: Confounder Detector Guidance for Improved Diffusion-based Counterfactual Explanations

Nima Fathi

Amar Kumar

Brennan Nichyporuk

Mohammad Havaei

Deep learning classifiers are prone to latching onto dominant confounders present in a dataset rather than on the causal markers associated … (voir plus)with the target class, leading to poor generalization and biased predictions. Although explainability via counterfactual image generation has been successful at exposing the problem, bias mitigation strategies that permit accurate explainability in the presence of dominant and diverse artifacts remain unsolved. In this work, we propose the DeCoDEx framework and show how an external, pre-trained binary artifact detector can be leveraged during inference to guide a diffusion-based counterfactual image generator towards accurate explainability. Experiments on the CheXpert dataset, using both synthetic artifacts and real visual artifacts (support devices), show that the proposed method successfully synthesizes the counterfactual images that change the causal pathology markers associated with Pleural Effusion while preserving or ignoring the visual artifacts. Augmentation of ERM and Group-DRO classifiers with the DeCoDEx generated images substantially improves the results across underrepresented groups that are out of distribution for each class. The code is made publicly available at https://github.com/NimaFathi/DeCoDEx.

2024-02-13

MIDL.io/2024/Conference/Submission (inconnu)

Metrics reloaded: Pitfalls and recommendations for image analysis validation

Lena Maier-Hein

Annika Reinke

Evangelia Christodoulou

Ben Glocker

Patrick Godau

Fabian Isensee

Jens Kleesiek

Michal Kozubek

Mauricio Reyes

Michael A. Riegler

Manuel Wiesenfarth

Michael Baumgartner

Matthias Eisenmann

Doreen Heckmann-Notzel

A. Emre Kavu

Tim Radsch

Minu Dietlinde Tizabi

Laura C. Acion

Michela Antonelli

Tal Arbel … (voir 47 de plus)

Spyridon Bakas

Peter Bankhead

Allison Benis

M. Cardoso

Veronika Cheplygina

Beth A. Cimini

Gary S. Collins

Keyvan Farahani

Bram van Ginneken

Daniel A. Hashimoto

Michael M. Hoffman

Merel Huisman

Pierre Jannin

Charles E. Kahn

A. Karargyris

Alan Karthikesalingam

H. Kenngott

Annette Kopp-Schneider

Anna Kreshuk

Tahsin Kurc

Bennett Landman

G. Litjens

Amin Madani

Klaus Maier-Hein

Anne L. Martel

Peter Mattson

Erik H. W. Meijering

Bjoern Menze

David Moher

K. Moons

Henning Müller

Felix Nickel

Brennan Nichyporuk

Jens Petersen

Nasir M. Rajpoot

Nicola Rieke

Julio Saez-Rodriguez

Clarisa S'anchez Guti'errez

Shravya Jaganath Shetty

M. Smeden

Carole H. Sudre

Ronald M. Summers

Abdel Aziz Taha

Sotirios A. Tsaftaris

B. Calster

Gael Varoquaux

Paul F. Jäger

2024-02-12

Nature Methods (publié)

Current AI applications in neurology: Brain imaging

Joshua D. Durso-Finley

Jean-Pierre R. Falet

Raghav Mehta

Douglas Arnold

Nick Pawlowski

2023-12-01

Journal of the Neurological Sciences (publié)

Improving Robustness and Reliability in Medical Image Classification with Latent-Guided Diffusion and Nested-Ensembles

Xing Shen

Hengguan Huang

Brennan Nichyporuk

2023-10-24

ArXiv (prépublication)

Debiasing Counterfactuals in the Presence of Spurious Correlations

Amar Kumar

Nima Fathi

Raghav Mehta

Brennan Nichyporuk

Jean-Pierre R. Falet

Sotirios A. Tsaftaris

Deep learning models can perform well in complex medical imaging classification tasks, even when basing their conclusions on spurious correl… (voir plus)ations (i.e. confounders), should they be prevalent in the training dataset, rather than on the causal image markers of interest. This would thereby limit their ability to generalize across the population. Explainability based on counterfactual image generation can be used to expose the confounders but does not provide a strategy to mitigate the bias. In this work, we introduce the first end-to-end training framework that integrates both (i) popular debiasing classifiers (e.g. distributionally robust optimization (DRO)) to avoid latching onto the spurious correlations and (ii) counterfactual image generation to unveil generalizable imaging markers of relevance to the task. Additionally, we propose a novel metric, Spurious Correlation Latching Score (SCLS), to quantify the extent of the classifier reliance on the spurious correlation as exposed by the counterfactual images. Through comprehensive experiments on two public datasets (with the simulated and real visual artifacts), we demonstrate that the debiasing method: (i) learns generalizable markers across the population, and (ii) successfully ignores spurious correlations and focuses on the underlying disease pathology.

2023-10-09

Clinical Image-Based Procedures, Fairness of AI in Medical Imaging, and Ethical and Philosophical Issues in Medical Imaging (publié)

Improving Image-Based Precision Medicine with Uncertainty-Aware Causal Models

Joshua D. Durso-Finley

Jean-Pierre R. Falet

Raghav Mehta

Douglas Arnold

Nick Pawlowski

Image-based precision medicine aims to personalize treatment decisions based on an individual's unique imaging features so as to improve the… (voir plus)ir clinical outcome. Machine learning frameworks that integrate uncertainty estimation as part of their treatment recommendations would be safer and more reliable. However, little work has been done in adapting uncertainty estimation techniques and validation metrics for precision medicine. In this paper, we use Bayesian deep learning for estimating the posterior distribution over factual and counterfactual outcomes on several treatments. This allows for estimating the uncertainty for each treatment option and for the individual treatment effects (ITE) between any two treatments. We train and evaluate this model to predict future new and enlarging T2 lesion counts on a large, multi-center dataset of MR brain images of patients with multiple sclerosis, exposed to several treatments during randomized controlled trials. We evaluate the correlation of the uncertainty estimate with the factual error, and, given the lack of ground truth counterfactual outcomes, demonstrate how uncertainty for the ITE prediction relates to bounds on the ITE error. Lastly, we demonstrate how knowledge of uncertainty could modify clinical decision-making to improve individual patient and clinical trial outcomes.

2023-10-08

OpenReview.net/Archive (publié)

Mitigating Calibration Bias Without Fixed Attribute Grouping for Improved Fairness in Medical Imaging Analysis

Changjian Shui

Justin Szeto

Raghav Mehta

Douglas Arnold

2023-10-08

OpenReview.net/Archive (publié)

Grow-push-prune: Aligning deep discriminants for effective structural network compression

Qing Tian

James J. Clark

2023-06-01

Computer Vision and Image Understanding (publié)

Medical SAM Adapter: Adapting Segment Anything Model for Medical Image Segmentation

Junde Wu

Rao Fu

Huihui Fang

Yuanpei Liu

Zhao-Yang Wang

Yanwu Xu

Yueming Jin

The Segment Anything Model (SAM) has recently gained popularity in the field of image segmentation due to its impressive capabilities in var… (voir plus)ious segmentation tasks and its prompt-based interface. However, recent studies and individual experiments have shown that SAM underperforms in medical image segmentation, since the lack of the medical specific knowledge. This raises the question of how to enhance SAM's segmentation capability for medical images. In this paper, instead of fine-tuning the SAM model, we propose the Medical SAM Adapter (Med-SA), which incorporates domain-specific medical knowledge into the segmentation model using a light yet effective adaptation technique. In Med-SA, we propose Space-Depth Transpose (SD-Trans) to adapt 2D SAM to 3D medical images and Hyper-Prompting Adapter (HyP-Adpt) to achieve prompt-conditioned adaptation. We conduct comprehensive evaluation experiments on 17 medical image segmentation tasks across various image modalities. Med-SA outperforms several state-of-the-art (SOTA) medical image segmentation methods, while updating only 2\% of the parameters. Our code is released at https://github.com/KidsWithTokens/Medical-SAM-Adapter.

2023-04-25

ArXiv (prépublication)

Evaluating the Fairness of Deep Learning Uncertainty Estimates in Medical Image Analysis

Raghav Mehta

Changjian Shui

Although deep learning (DL) models have shown great success in many medical image analysis tasks, deployment of the resulting models into r… (voir plus)eal clinical contexts requires: (1) that they exhibit robustness and fairness across different sub-populations, and (2) that the confidence in DL model predictions be accurately expressed in the form of uncertainties. Unfortunately, recent studies have indeed shown significant biases in DL models across demographic subgroups (e.g., race, sex, age) in the context of medical image analysis, indicating a lack of fairness in the models. Although several methods have been proposed in the ML literature to mitigate a lack of fairness in DL models, they focus entirely on the absolute performance between groups without considering their effect on uncertainty estimation. In this work, we present the first exploration of the effect of popular fairness models on overcoming biases across subgroups in medical image analysis in terms of bottom-line performance, and their effects on uncertainty quantification. We perform extensive experiments on three different clinically relevant tasks: (i) skin lesion classification, (ii) brain tumour segmentation, and (iii) Alzheimer's disease clinical score regression. Our results indicate that popular ML methods, such as data-balancing and distributionally robust optimization, succeed in mitigating fairness issues in terms of the model performances for some of the tasks. However, this can come at the cost of poor uncertainty estimates associated with the model predictions. This tradeoff must be mitigated if fairness models are to be adopted in medical image analysis.

2023-04-04

MIDL.io/2023/Conference (poster)

Understanding metric-related pitfalls in image analysis validation

Annika Reinke

Minu Dietlinde Tizabi

Michael Baumgartner

Matthias Eisenmann

Doreen Heckmann-Notzel

A. Emre Kavu

Tim Radsch

Carole H. Sudre

Laura C. Acion

Michela Antonelli

Spyridon Bakas

Allison Benis

Arriel Benis

Matthew B. Blaschko

Florian Buettner

Florian Buttner

M. Cardoso

Veronika Cheplygina

Jianxu Chen … (voir 62 de plus)

Evangelia Christodoulou

Beth A. Cimini

Keyvan Farahani

Luciana Ferrer

Gary S. Collins

Adrian Galdran

Bram van Ginneken

Ben Glocker

Patrick Godau

Daniel A. Hashimoto

Michael M. Hoffman

Robert Cary Haase

Merel Huisman

Fabian Isensee

Pierre Jannin

Charles E. Kahn

Dagmar Kainmueller

Bernhard Kainz

A. Karargyris

Jens Kleesiek

Florian Kofler

Thijs Kooi

Annette Kopp-Schneider

Alan Karthikesalingam

H. Kenngott

Michal Kozubek

Anna Kreshuk

Tahsin Kurc

Bennett A. Landman

G. Litjens

Amin Madani

Klaus Maier-Hein

Anne L. Martel

Erik H. W. Meijering

Bjoern Menze

K. Moons

Henning Müller

Brennan Nichyporuk

Peter Mattson

Felix Nickel

Jens Petersen

Susanne M. Rafelski

Nasir M. Rajpoot

Mauricio Reyes

Michael A. Riegler

Nicola Rieke

Julio Saez-Rodriguez

Clara I. Sánchez

Shravya Jaganath Shetty

Ronald M. Summers

Abdel Aziz Taha

Aleksei Tiulpin

Sotirios A. Tsaftaris

Ben Van Calster

Gael Varoquaux

Ziv R Yaniv

M. Smeden

Paul F. Jäger

Lena Maier-Hein

B. Calster

Manuel Wiesenfarth

Ziv Rafael Yaniv

2023-02-09

arXiv.org (prépublication)