Brennan Nichyporuk

Spatio-Temporal Conditional Diffusion Models for Forecasting Future Multiple Sclerosis Lesion Masks Conditioned on Treatments

Gian Mario Favero

Ge Ya Luo

Douglas Arnold

2025-08-09

ArXiv (preprint)

Conditional Diffusion Models are Medical Image Classifiers that Provide Explainability and Uncertainty for Free

Gian Mario Favero

2025-03-27

MIDL.io/2025/Conference (oral)

Conditional Diffusion Models are Medical Image Classifiers that Provide Explainability and Uncertainty for Free

Gian Mario Favero

Discriminative classifiers have become a foundational tool in deep learning for medical imaging, excelling at learning separable features of… (see more) complex data distributions. However, these models often need careful design, augmentation, and training techniques to ensure safe and reliable deployment. Recently, diffusion models have become synonymous with generative modeling in 2D. These models showcase robustness across a range of tasks including natural image classification, where classification is performed by comparing reconstruction errors across images generated for each possible conditioning input. This work presents the first exploration of the potential of class conditional diffusion models for 2D medical image classification. First, we develop a novel majority voting scheme shown to improve the performance of medical diffusion classifiers. Next, extensive experiments on the CheXpert and ISIC Melanoma skin cancer datasets demonstrate that foundation and trained-from-scratch diffusion models achieve competitive performance against SOTA discriminative classifiers without the need for explicit supervision. In addition, we show that diffusion classifiers are intrinsically explainable, and can be used to quantify the uncertainty of their predictions, increasing their trustworthiness and reliability in safety-critical, clinical contexts. Further information is available on our project page: https://faverogian.github.io/med-diffusion-classifier.github.io/

2025-02-06

ArXiv (preprint)

Improving Robustness and Reliability in Medical Image Classification with Latent-Guided Diffusion and Nested-Ensembles

Xing Shen

Hengguan Huang

2025-01-01

IEEE Transactions on Medical Imaging (published)

Exploring Compound Loss Functions for Brain Tumor Segmentation

Anita Kriz

Raghav Mehta

2024-12-28

Lecture Notes in Computer Science (published)

HyperFusion: A Hypernetwork Approach to Multimodal Integration of Tabular and Medical Imaging Data for Predictive Modeling

Daniel Duenias

Tammy Riklin-Raviv

The integration of diverse clinical modalities such as medical imaging and the tabular data obtained by the patients' Electronic Health Reco… (see more)rds (EHRs) is a crucial aspect of modern healthcare. The integrative analysis of multiple sources can provide a comprehensive understanding of a patient's condition and can enhance diagnoses and treatment decisions. Deep Neural Networks (DNNs) consistently showcase outstanding performance in a wide range of multimodal tasks in the medical domain. However, the complex endeavor of effectively merging medical imaging with clinical, demographic and genetic information represented as numerical tabular data remains a highly active and ongoing research pursuit. We present a novel framework based on hypernetworks to fuse clinical imaging and tabular data by conditioning the image processing on the EHR's values and measurements. This approach aims to leverage the complementary information present in these modalities to enhance the accuracy of various medical applications. We demonstrate the strength and the generality of our method on two different brain Magnetic Resonance Imaging (MRI) analysis tasks, namely, brain age prediction conditioned by subject's sex, and multiclass Alzheimer's Disease (AD) classification conditioned by tabular data. We show that our framework outperforms both single-modality models and state-of-the-art MRI-tabular data fusion methods. The code, enclosed to this manuscript will be made publicly available.

2024-03-20

ArXiv (preprint)

DeCoDEx: Confounder Detector Guidance for Improved Diffusion-based Counterfactual Explanations

Mohammad Havaei

Deep learning classifiers are prone to latching onto dominant confounders present in a dataset rather than on the causal markers associated … (see more)with the target class, leading to poor generalization and biased predictions. Although explainability via counterfactual image generation has been successful at exposing the problem, bias mitigation strategies that permit accurate explainability in the presence of dominant and diverse artifacts remain unsolved. In this work, we propose the DeCoDEx framework and show how an external, pre-trained binary artifact detector can be leveraged during inference to guide a diffusion-based counterfactual image generator towards accurate explainability. Experiments on the CheXpert dataset, using both synthetic artifacts and real visual artifacts (support devices), show that the proposed method successfully synthesizes the counterfactual images that change the causal pathology markers associated with Pleural Effusion while preserving or ignoring the visual artifacts. Augmentation of ERM and Group-DRO classifiers with the DeCoDEx generated images substantially improves the results across underrepresented groups that are out of distribution for each class. The code is made publicly available at https://github.com/NimaFathi/DeCoDEx.

2024-02-13

MIDL.io/2024/Conference/Submission (unknown)

Metrics reloaded: Pitfalls and recommendations for image analysis validation

Lena Maier-Hein

Annika Reinke

Evangelia Christodoulou

Ben Glocker

PATRICK GODAU

Fabian Isensee

Jens Kleesiek

Michal Kozubek

Mauricio Reyes

MICHAEL A. RIEGLER

Manuel Wiesenfarth

Michael Baumgartner

Matthias Eisenmann

DOREEN HECKMANN-NÖTZEL

A. EMRE KAVUR

TIM RÄDSCH

Minu Dietlinde Tizabi

LAURA ACION

Michela Antonelli

Tal Arbel … (see 47 more)

Spyridon Bakas

Peter Bankhead

Allison Benis

M. Jorge Cardoso

Veronika Cheplygina

BETH A. CIMINI

Gary S. Collins

Keyvan Farahani

Bram van Ginneken

Daniel A. Hashimoto

Michael M. Hoffman

Merel Huisman

Pierre Jannin

CHARLES E. KAHN

Alexandros Karargyris

Alan Karthikesalingam

H. Kenngott

Annette Kopp-Schneider

Anna Kreshuk

Tahsin Kurc

Bennett Landman

GEERT LITJENS

Amin Madani

Klaus Maier-Hein

Anne L. Martel

Peter Mattson

ERIK MEIJERING

Bjoern Menze

David Moher

KAREL G.M. MOONS

Henning Müller

Felix Nickel

Jens Petersen

Nasir Rajpoot

Nicola Rieke

Julio Saez-Rodriguez

Clarisa S'anchez Guti'errez

SHRAVYA SHETTY

M. Smeden

Carole H. Sudre

Ronald M. Summers

Abdel Aziz Taha

Sotirios A. Tsaftaris

Ben Van Calster

Gael Varoquaux

PAUL F. JÄGER

2024-02-12

Nature Methods (published)

Understanding metric-related pitfalls in image analysis validation

Annika Reinke

Minu Dietlinde Tizabi

Michael Baumgartner

Matthias Eisenmann

DOREEN HECKMANN-NÖTZEL

A. EMRE KAVUR

TIM RÄDSCH

Carole H. Sudre

LAURA ACION

Michela Antonelli

Spyridon Bakas

Allison Benis

Arriel Benis

Matthew Blaschko

FLORIAN BUETTNER

M. Jorge Cardoso

Veronika Cheplygina

JIANXU CHEN

Evangelia Christodoulou … (see 59 more)

BETH A. CIMINI

Gary S. Collins

Keyvan Farahani

LUCIANA FERRER

Adrian Galdran

Bram van Ginneken

Ben Glocker

PATRICK GODAU

Robert Cary Haase

Daniel A. Hashimoto

Michael M. Hoffman

Merel Huisman

Fabian Isensee

Pierre Jannin

CHARLES E. KAHN

DAGMAR KAINMUELLER

BERNHARD KAINZ

Alexandros Karargyris

Alan Karthikesalingam

H. Kenngott

Jens Kleesiek

Florian Kofler

THIJS KOOI

Annette Kopp-Schneider

Michal Kozubek

Anna Kreshuk

Tahsin Kurc

BENNETT A. LANDMAN

GEERT LITJENS

Amin Madani

Klaus Maier-Hein

Anne L. Martel

Peter Mattson

ERIK MEIJERING

Bjoern Menze

KAREL G.M. MOONS

Henning Müller

Felix Nickel

Jens Petersen

SUSANNE M. RAFELSKI

Nasir Rajpoot

Mauricio Reyes

MICHAEL A. RIEGLER

Nicola Rieke

Julio Saez-Rodriguez

Clara I. Sánchez

SHRAVYA SHETTY

M. Smeden

Ronald M. Summers

Abdel Aziz Taha

ALEKSEI TIULPIN

Sotirios A. Tsaftaris

Ben Van Calster

Gael Varoquaux

Manuel Wiesenfarth

ZIV R. YANIV

PAUL F. JÄGER

Lena Maier-Hein

2024-02-12

Nature methods (published)

DeCoDEx: Confounder Detector Guidance for Improved Diffusion-based Counterfactual Explanations

Mohammad Havaei

Deep learning classifiers are prone to latching onto dominant confounders present in a dataset rather than on the causal markers associated … (see more)with the target class, leading to poor generalization and biased predictions. Although explainability via counterfactual image generation has been successful at exposing the problem, bias mitigation strategies that permit accurate explainability in the presence of dominant and diverse artifacts remain unsolved. In this work, we propose the DeCoDEx framework and show how an external, pre-trained binary artifact detector can be leveraged during inference to guide a diffusion-based counterfactual image generator towards accurate explainability. Experiments on the CheXpert dataset, using both synthetic artifacts and real visual artifacts (support devices), show that the proposed method successfully synthesizes the counterfactual images that change the causal pathology markers associated with Pleural Effusion while preserving or ignoring the visual artifacts. Augmentation of ERM and Group-DRO classifiers with the DeCoDEx generated images substantially improves the results across underrepresented groups that are out of distribution for each class. The code is made publicly available at https://github.com/NimaFathi/DeCoDEx.

2024-01-01

MIDL (published)

Debiasing Counterfactuals in the Presence of Spurious Correlations

Amar Kumar

Nima Fathi

Raghav Mehta

Jean-Pierre R. Falet

Sotirios A. Tsaftaris

Deep learning models can perform well in complex medical imaging classification tasks, even when basing their conclusions on spurious correl… (see more)ations (i.e. confounders), should they be prevalent in the training dataset, rather than on the causal image markers of interest. This would thereby limit their ability to generalize across the population. Explainability based on counterfactual image generation can be used to expose the confounders but does not provide a strategy to mitigate the bias. In this work, we introduce the first end-to-end training framework that integrates both (i) popular debiasing classifiers (e.g. distributionally robust optimization (DRO)) to avoid latching onto the spurious correlations and (ii) counterfactual image generation to unveil generalizable imaging markers of relevance to the task. Additionally, we propose a novel metric, Spurious Correlation Latching Score (SCLS), to quantify the extent of the classifier reliance on the spurious correlation as exposed by the counterfactual images. Through comprehensive experiments on two public datasets (with the simulated and real visual artifacts), we demonstrate that the debiasing method: (i) learns generalizable markers across the population, and (ii) successfully ignores spurious correlations and focuses on the underlying disease pathology.

2023-10-09

Clinical Image-Based Procedures, Fairness of AI in Medical Imaging, and Ethical and Philosophical Issues in Medical Imaging (published)