Tal Arbel

Biography

Tal Arbel is a professor in the Department of Electrical and Computer Engineering at McGill University, where she is the director of the Probabilistic Vision Group and Medical Imaging Lab in the Centre for Intelligent Machines.

She is also a Canada CIFAR AI Chair, an associate academic member of Mila – Quebec Artificial Intelligence Institute and an associate member of the Goodman Cancer Research Centre.

Arbel’s research focuses on the development of probabilistic deep learning methods in computer vision and medical image analysis for a wide range of real-world applications, with a focus on neurological diseases.

She is a recipient of the 2019 McGill Engineering Christophe Pierre Research Award and a Fellow of the Canadian Academy of Engineering. She regularly serves on the organizing team of major international conferences in computer vision and in medical image analysis (e.g. MICCAI, MIDL, ICCV, CVPR). She is currently the Editor-in-Chief and co-founder of the arXiv overlay journal: Machine Learning for Biomedical Imaging (MELBA).

Current Students

Karl Bridi

Research Intern - McGill University

Cornelius Crijnen

PhD - McGill University

Website

Oscar Cruzhernandez

Research Intern - McGill University

Charbel El Feghali

Research Intern - McGill University

Carlotta Hoelzle

Research Intern - McGill University

Elizabeth Laura Janes

Master's Research - McGill University

Emily Kaczmarek

PhD - McGill University

Yusong Li

Collaborating researcher - McGill University University

Toky Raharison Ralambomihanta

Yik Yu Ng

Research Intern - McGill University

Research Intern - McGill University

Ryan Rezai

Master's Research - McGill University

Rachel Ruddy

Research Intern - McGill University

Minh To

Collaborating researcher - UBC

Website

PRISM: An Explainable Generative AI Model for Medical Imaging

Blog Posts

Image of an Xray and the DDIM process to generate counterfactual version of Xrays

July 1, 2025

Amar Kumar

Anita Kriz

Mohammed Havaei

Tal Arbel

Read the article

Publications

Metrics reloaded: recommendations for image analysis validation.

Lena Maier-Hein

Annika Reinke

Evangelia Christodoulou

Ben Glocker

PATRICK GODAU

Fabian Isensee

Jens Kleesiek

Michal Kozubek

Mauricio Reyes

MICHAEL A. RIEGLER

Manuel Wiesenfarth

Michael Baumgartner

Matthias Eisenmann

DOREEN HECKMANN-NÖTZEL

A. EMRE KAVUR

TIM RÄDSCH

Minu Dietlinde Tizabi

Laura Acion

Michela Antonelli

Tal Arbel … (see 47 more)

Spyridon Bakas

Peter Bankhead

Allison Benis

M. Jorge Cardoso

Veronika Cheplygina

BETH A. CIMINI

Gary S. Collins

Keyvan Farahani

Bram van Ginneken

Daniel A. Hashimoto

Michael M. Hoffman

Merel Huisman

Pierre Jannin

CHARLES E. KAHN

Alexandros Karargyris

Alan Karthikesalingam

H. Kenngott

Annette Kopp-Schneider

Anna Kreshuk

Tahsin Kurc

Bennett Landman

GEERT LITJENS

Amin Madani

Klaus Maier-Hein

Anne L. Martel

Peter Mattson

Erik Meijering

Bjoern Menze

David Moher

Karel G.M. Moons

Henning Müller

Felix Nickel

Jens Petersen

Nasir Rajpoot

Nicola Rieke

Julio Saez-Rodriguez

Clarisa S'anchez Guti'errez

Shravya Shetty

M. Smeden

Carole H. Sudre

Ronald M. Summers

Abdel Aziz Taha

Sotirios A. Tsaftaris

Ben Van Calster

Gael Varoquaux

PAUL F. JÄGER

2024-02-11

Nature Methods (published)

Understanding metric-related pitfalls in image analysis validation

Annika Reinke

Minu D. Tizabi

Michael Baumgartner

Matthias Eisenmann

DOREEN HECKMANN-NÖTZEL

A. EMRE KAVUR

TIM RÄDSCH

Carole H. Sudre

Laura Acion

Michela Antonelli

Spyridon Bakas

Arriel Benis

Matthew Blaschko

Florian Buettner

M. Jorge Cardoso

Veronika Cheplygina

Jianxu Chen

Evangelia Christodoulou

BETH A. CIMINI … (see 58 more)

Gary S. Collins

Keyvan Farahani

LUCIANA FERRER

Adrian Galdran

Bram van Ginneken

Ben Glocker

PATRICK GODAU

Robert Haase

Daniel A. Hashimoto

Michael M. Hoffman

Merel Huisman

Fabian Isensee

Pierre Jannin

CHARLES E. KAHN

Dagmar Kainmueller

BERNHARD KAINZ

Alexandros Karargyris

Alan Karthikesalingam

Hannes Kenngott

Jens Kleesiek

Florian Kofler

Thijs Kooi

Annette Kopp-Schneider

Michal Kozubek

Anna Kreshuk

Tahsin Kurc

Bennett A. Landman

GEERT LITJENS

Amin Madani

Klaus Maier-Hein

Anne L. Martel

Peter Mattson

Erik Meijering

Bjoern Menze

Karel G. M. Moons

Henning Müller

Felix Nickel

Jens Petersen

Susanne M. Rafelski

Nasir Rajpoot

Mauricio Reyes

MICHAEL A. RIEGLER

Nicola Rieke

Julio Saez-Rodriguez

Clara I. Sánchez

Shravya Shetty

Maarten van Smeden

Ronald M. Summers

Abdel A. Taha

Aleksei Tiulpin

Sotirios A. Tsaftaris

Ben Van Calster

Gael Varoquaux

Manuel Wiesenfarth

Ziv R. Yaniv

PAUL F. JÄGER

Lena Maier-Hein

Validation metrics are key for the reliable tracking of scientific progress and for bridging the current chasm between artificial intelligen… (see more)ce (AI) research and its translation into practice. However, increasing evidence shows that particularly in image analysis, metrics are often chosen inadequately in relation to the underlying research problem. This could be attributed to a lack of accessibility of metric-related knowledge: While taking into account the individual strengths, weaknesses, and limitations of validation metrics is a critical prerequisite to making educated choices, the relevant knowledge is currently scattered and poorly accessible to individual researchers. Based on a multi-stage Delphi process conducted by a multidisciplinary expert consortium as well as extensive community feedback, the present work provides the first reliable and comprehensive common point of access to information on pitfalls related to validation metrics in image analysis. Focusing on biomedical image analysis but with the potential of transfer to other fields, the addressed pitfalls generalize across application domains and are categorized according to a newly created, domain-agnostic taxonomy. To facilitate comprehension, illustrations and specific examples accompany each pitfall. As a structured body of information accessible to researchers of all levels of expertise, this work enhances global comprehension of a key topic in image analysis validation.

2024-01-31

Nature Methods (published)

DeCoDEx: Confounder Detector Guidance for Improved Diffusion-based Counterfactual Explanations

Deep learning classifiers are prone to latching onto dominant confounders present in a dataset rather than on the causal markers associated … (see more)with the target class, leading to poor generalization and biased predictions. Although explainability via counterfactual image generation has been successful at exposing the problem, bias mitigation strategies that permit accurate explainability in the presence of dominant and diverse artifacts remain unsolved. In this work, we propose the DeCoDEx framework and show how an external, pre-trained binary artifact detector can be leveraged during inference to guide a diffusion-based counterfactual image generator towards accurate explainability. Experiments on the CheXpert dataset, using both synthetic artifacts and real visual artifacts (support devices), show that the proposed method successfully synthesizes the counterfactual images that change the causal pathology markers associated with Pleural Effusion while preserving or ignoring the visual artifacts. Augmentation of ERM and Group-DRO classifiers with the DeCoDEx generated images substantially improves the results across underrepresented groups that are out of distribution for each class. The code is made publicly available at https://github.com/NimaFathi/DeCoDEx.

2023-12-31

MIDL (published)

HyperFusion: A Hypernetwork Approach to Multimodal Integration of Tabular and Medical Imaging Data for Predictive Modeling

Daniel Duenias

Tammy Riklin Raviv

The integration of diverse clinical modalities such as medical imaging and the tabular data extracted from patients' Electronic Health Recor… (see more)ds (EHRs) is a crucial aspect of modern healthcare. Integrative analysis of multiple sources can provide a comprehensive understanding of the clinical condition of a patient, improving diagnosis and treatment decision. Deep Neural Networks (DNNs) consistently demonstrate outstanding performance in a wide range of multimodal tasks in the medical domain. However, the complex endeavor of effectively merging medical imaging with clinical, demographic and genetic information represented as numerical tabular data remains a highly active and ongoing research pursuit. We present a novel framework based on hypernetworks to fuse clinical imaging and tabular data by conditioning the image processing on the EHR's values and measurements. This approach aims to leverage the complementary information present in these modalities to enhance the accuracy of various medical applications. We demonstrate the strength and generality of our method on two different brain Magnetic Resonance Imaging (MRI) analysis tasks, namely, brain age prediction conditioned by subject's sex and multi-class Alzheimer's Disease (AD) classification conditioned by tabular data. We show that our framework outperforms both single-modality models and state-of-the-art MRI tabular data fusion methods. A link to our code can be found at https://github.com/daniel4725/HyperFusion

2023-12-31

arXiv (preprint)

Current AI applications in neurology: Brain imaging

Joshua D. Durso-Finley

Jean-Pierre R. Falet

Raghav Mehta

Douglas Arnold

Nick Pawlowski

2023-11-30

Journal of the Neurological Sciences (published)

Debiasing Counterfactuals in the Presence of Spurious Correlations

Amar Kumar

Nima Fathi

Raghav Mehta

Jean-Pierre R. Falet

Sotirios A. Tsaftaris

Deep learning models can perform well in complex medical imaging classification tasks, even when basing their conclusions on spurious correl… (see more)ations (i.e. confounders), should they be prevalent in the training dataset, rather than on the causal image markers of interest. This would thereby limit their ability to generalize across the population. Explainability based on counterfactual image generation can be used to expose the confounders but does not provide a strategy to mitigate the bias. In this work, we introduce the first end-to-end training framework that integrates both (i) popular debiasing classifiers (e.g. distributionally robust optimization (DRO)) to avoid latching onto the spurious correlations and (ii) counterfactual image generation to unveil generalizable imaging markers of relevance to the task. Additionally, we propose a novel metric, Spurious Correlation Latching Score (SCLS), to quantify the extent of the classifier reliance on the spurious correlation as exposed by the counterfactual images. Through comprehensive experiments on two public datasets (with the simulated and real visual artifacts), we demonstrate that the debiasing method: (i) learns generalizable markers across the population, and (ii) successfully ignores spurious correlations and focuses on the underlying disease pathology.

2023-10-08

Clinical Image-Based Procedures, Fairness of AI in Medical Imaging, and Ethical and Philosophical Issues in Medical Imaging (published)

Improving Image-Based Precision Medicine with Uncertainty-Aware Causal Models

Joshua D. Durso-Finley

Jean-Pierre R. Falet

Raghav Mehta

Douglas Arnold

Nick Pawlowski

Image-based precision medicine aims to personalize treatment decisions based on an individual's unique imaging features so as to improve the… (see more)ir clinical outcome. Machine learning frameworks that integrate uncertainty estimation as part of their treatment recommendations would be safer and more reliable. However, little work has been done in adapting uncertainty estimation techniques and validation metrics for precision medicine. In this paper, we use Bayesian deep learning for estimating the posterior distribution over factual and counterfactual outcomes on several treatments. This allows for estimating the uncertainty for each treatment option and for the individual treatment effects (ITE) between any two treatments. We train and evaluate this model to predict future new and enlarging T2 lesion counts on a large, multi-center dataset of MR brain images of patients with multiple sclerosis, exposed to several treatments during randomized controlled trials. We evaluate the correlation of the uncertainty estimate with the factual error, and, given the lack of ground truth counterfactual outcomes, demonstrate how uncertainty for the ITE prediction relates to bounds on the ITE error. Lastly, we demonstrate how knowledge of uncertainty could modify clinical decision-making to improve individual patient and clinical trial outcomes.

2023-10-07

OpenReview.net/Archive (published)

Mitigating Calibration Bias Without Fixed Attribute Grouping for Improved Fairness in Medical Imaging Analysis

Changjian Shui

Justin Szeto

Raghav Mehta

Douglas L. Arnold

2023-10-07

OpenReview (published)

Grow-push-prune: Aligning deep discriminants for effective structural network compression

Qing Tian

James J. Clark

2023-05-31

Computer Vision and Image Understanding (published)

Medical SAM Adapter: Adapting Segment Anything Model for Medical Image Segmentation

Junde Wu

Rao Fu

Huihui Fang

Yuanpei Liu

Zhao-Yang Wang

Yanwu Xu

Yueming Jin

The Segment Anything Model (SAM) has recently gained popularity in the field of image segmentation due to its impressive capabilities in var… (see more)ious segmentation tasks and its prompt-based interface. However, recent studies and individual experiments have shown that SAM underperforms in medical image segmentation, since the lack of the medical specific knowledge. This raises the question of how to enhance SAM's segmentation capability for medical images. In this paper, instead of fine-tuning the SAM model, we propose the Medical SAM Adapter (Med-SA), which incorporates domain-specific medical knowledge into the segmentation model using a light yet effective adaptation technique. In Med-SA, we propose Space-Depth Transpose (SD-Trans) to adapt 2D SAM to 3D medical images and Hyper-Prompting Adapter (HyP-Adpt) to achieve prompt-conditioned adaptation. We conduct comprehensive evaluation experiments on 17 medical image segmentation tasks across various image modalities. Med-SA outperforms several state-of-the-art (SOTA) medical image segmentation methods, while updating only 2\% of the parameters. Our code is released at https://github.com/KidsWithTokens/Medical-SAM-Adapter.

2023-04-24

ArXiv (preprint)

Evaluating the Fairness of Deep Learning Uncertainty Estimates in Medical Image Analysis

Raghav Mehta

Changjian Shui

2023-04-03

MIDL.io/2023/Conference (poster)

proceedings.mlr.press

Personalized Prediction of Future Lesion Activity and Treatment Effect in Multiple Sclerosis from Baseline MRI

Joshua D. Durso-Finley

Jean-Pierre R. Falet

Douglas Arnold

Precision medicine for chronic diseases such as multiple sclerosis (MS) involves choosing a treatment which best balances efficacy and side … (see more)effects/preferences for individual patients. Making this choice as early as possible is important, as delays in finding an effective therapy can lead to irreversible disability accrual. To this end, we present the first deep neural network model for individualized treatment decisions from baseline magnetic resonance imaging (MRI) (with clinical information if available) for MS patients which (a) predicts future new and enlarging T2 weighted (NE-T2) lesion counts on follow-up MRI on multiple treatments and (b) estimates the conditional average treatment effect (CATE), as defined by the predicted future suppression of NE-T2 lesions, between different treatment options relative to placebo. Our model is validated on a proprietary federated dataset of 1817 multi-sequence MRIs acquired from MS patients during four multi-centre randomized clinical trials. Our framework achieves high average precision in the binarized regression of future NE-T2 lesions on five different treatments, identifies heterogeneous treatment effects, and provides a personalized treatment recommendation that accounts for treatment-associated risk (side effects, patient preference, administration difficulties,...).

2022-12-03

Proceedings of The 5th International Conference on Medical Imaging with Deep Learning (published)