Brennan Nichyporuk

Jorge Corral Acero

Sujit Kumar Das

Marcela de Oliveira

Hanqiu Deng

Guiming Dong

Lars Doorenbos

Cory Efird

Di Fan

Mehdi Fatan Serj

Alexandre Fenneteau

Lucas Fidon

Patryk Filipiak

Ren'e Finzel

Nuno Renato Freitas

C. Friedrich

Mitchell J. Fulton

Finn Gaida

Francesco Galati

Christoforos Galazis

Changna Gan

Zheyao Gao

Sheng Gao

Matej Gazda

Beerend G. A. Gerats

Neil Getty

Adam Gibicar

Ryan J. Gifford

Sajan Gohil

Maria Grammatikopoulou

Daniel Grzech

Orhun Guley

Timo Gunnemann

Chun-Hai Guo

Sylvain Guy

Heonjin Ha

Luyi Han

Ilseok Han

Ali Hatamizadeh

Tianhai He

Ji-Wu Heo

Sebastian Hitziger

SeulGi Hong

Seungbum Hong

Rian Huang

Zi-You Huang

Markus Huellebrand

Stephan Huschauer

M. Hussain

Tomoo Inubushi

Ece Isik Polat

Mojtaba Jafaritadi

Seonghun Jeong

Bailiang Jian

Yu Jiang

Zhifan Jiang

Yu Jin

Smriti Joshi

A. Kadkhodamohammadi

R. A. Kamraoui

Inhak Kang

Jun-Su Kang

Davood Karimi

April Ellahe Khademi

Muhammad Irfan Khan

Suleiman A. Khan

Rishab Khantwal

Kwang-Ju Kim

Timothy Lee Kline

Satoshi Kondo

Elina Kontio

Adrian Krenzer

Artem Kroviakov

Hugo J. Kuijf

Satyadwyoom Kumar

Francesco La Rosa

Abhishek Lad

Doohee Lee

Minho Lee

Chiara Lena

Hao Li

Ling Li

Xingyu Li

F. Liao

Kuan-Ya Liao

Arlindo L. Oliveira

Chaonan Lin

Shanhai Lin

Akis Linardos

M. Linguraru

Han Liu

Tao Liu

Dian Liu

Yanling Liu

Joao Lourencco-Silva

Jing Lu

Jia Lu

Imanol Luengo

Christina Bach Lund

Huan Minh Luu

Yingqi Lv

Leon Maechler

L. SinaMansour

Kenji Marshall

Moona Mazher

Richard McKinley

Alfonso Medela

Felix Meissen

Mingyuan Meng

Dylan Bradley Miller

S. Mirjahanmardi

Arnab Kumar Mishra

Samir Mitha

Hassan Mohy-ud-Din

Tony C. W. Mok

Gowtham Krishnan Murugesan

Enamundram Naga Karthik

Sahil Nalawade

Jakub Nalepa

M. Naser

Ramin Nateghi

Hammad Naveed

Quang-Minh Nguyen

Cuong Nguyen Quoc

Bruno Oliveira

David Owen

Jimut Bahan Pal

Junwen Pan

W. Pan

Winnie Pang

Bogyu Park

Vivek G. Pawar

K. Pawar

Michael Peven

Lena Philipp

Tomasz Pieciak

Szymon S Płotka

Marcel Plutat

Fattane Pourakpour

Domen Prelovznik

K. Punithakumar

Abdul Qayyum

Sandro Queir'os

Arman Rahmim

Salar Razavi

Jintao Ren

Mina Rezaei

Jonathan Adam Rico

ZunHyan Rieu

Markus Rink

Johannes Roth

Yusely Ruiz-gonzalez

Numan Saeed

Anindo Saha

Mostafa M. Sami Salem

Ricardo Sanchez-matilla

Kurt G Schilling

Weizhen Shao

Zhiqiang Shen

Ruize Shi

Pengcheng Shi

Daniel Sobotka

Th'eodore Soulier

Bella Specktor Fadida

D. Stoyanov

Timothy Sum Hon Mun

Xiao-Fu Sun

Rong Tao

Franz Thaler

Antoine Th'eberge

Felix Thielke

Helena R. Torres

K. Wahid

Jiacheng Wang

Yifei Wang

W. Wang

Xiong Jun Wang

Jianhui Wen

Ning Wen

Marek Wodziński

Yehong Wu

Fangfang Xia

Tianqi Xiang

Cheng Xiaofei

Lizhang Xu

Tingting Xue

Yu‐Xia Yang

Lingxian Yang

Kai Yao

Huifeng Yao

Amirsaeed Yazdani

Michael Yip

Hwa-Seong Yoo

Fereshteh Yousefirizi

Shu-Fen Yu

Lei Yu

Jonathan Zamora

Ramy Ashraf Zeineldin

Dewen Zeng

Jianpeng Zhang

Bokai Zhang

Jiapeng Zhang

Fangxi Zhang

Huahong Zhang

Zhongchen Zhao

Zixuan Zhao

Jia Zhao

Can Zhao

Q. Zheng

Yuheng Zhi

Ziqi Zhou

Baosheng Zou

Klaus Maier-Hein

PAUL F. JÄGER

Annette Kopp-Schneider

Lena Maier-Hein

The number of international benchmarking competitions is steadily increasing in various fields of machine learning (ML) research and practic… (see more)e. So far, however, little is known about the common practice as well as bottlenecks faced by the community in tackling the research questions posed. To shed light on the status quo of algorithm development in the specific field of biomedical imaging analysis, we designed an international survey that was issued to all participants of challenges conducted in conjunction with the IEEE ISBI 2021 and MICCAI 2021 conferences (80 competitions in total). The survey covered participants' expertise and working environments, their chosen strategies, as well as algorithm characteristics. A median of 72% challenge participants took part in the survey. According to our results, knowledge exchange was the primary incentive (70%) for participation, while the reception of prize money played only a minor role (16%). While a median of 80 working hours was spent on method development, a large portion of participants stated that they did not have enough time for method development (32%). 25% perceived the infrastructure to be a bottleneck. Overall, 94% of all solutions were deep learning-based. Of these, 84% were based on standard architectures. 43% of the respondents reported that the data samples (e.g., images) were too large to be processed at once. This was most commonly addressed by patch-based training (69%), downsampling (37%), and solving 3D analysis tasks as a series of 2D tasks. K-fold cross-validation on the training set was performed by only 37% of the participants and only 50% of the participants performed ensembling based on multiple identical models (61%) or heterogeneous models (39%). 48% of the respondents applied postprocessing steps.

2022-12-16

ArXiv (preprint)

Biomedical image analysis competitions: The state of current participation practice

Matthias Eisenmann

Annika Reinke

Vivienn Weru

Minu Dietlinde Tizabi

Fabian Isensee

T. Adler

PATRICK GODAU

Veronika Cheplygina

Michal Kozubek

Sharib Ali

Anubha Gupta

Jan. Kybic

Alison Professor Noble

Carlos Ortiz de Sol'orzano

Samiksha Pachade

Caroline Petitjean

Daniel Sage

Donglai Wei

Elizabeth Wilden

Deepak Alapatt … (see 334 more)

Vincent Andrearczyk

Ujjwal Baid

Spyridon Bakas

Niranjan Balu

Sophia Bano

Vivek Singh Bawa

Jorge Bernal

Sebastian Bodenstedt

Alessandro Casella

Jinwook Choi

Olivier Commowick

M. Daum

Adrien Depeursinge

Reuben Dorent

J. Egger

H. Eichhorn

Sandy Engelhardt

Melanie Ganz

Gabriel Girard

Lasse Donovan Hansen

Mattias Paul Heinrich

Nicholas Heller

Alessa Hering

Arnaud Huaulm'e

Hyunjeong Kim

Bennett Landman

Hongwei Bran Li

Jianning Li

Junfang Ma

Anne L. Martel

Carlos Mart'in-Isla

Bjoern Menze

Chinedu Innocent Nwoye

Valentin Oreiller

Nicolas Padoy

Sarthak Pati

Kelly Payette

Carole H. Sudre

K. V. Wijnen

Armine Vardazaryan

Tom Kamiel Magda Vercauteren

Martin Wagner

Chuanbo Wang

Moi Hoon Yap

Zeyun Yu

Chuner Yuan

Maximilian Zenk

Aneeq Zia

David Zimmerer

Rina Bao

Chanyeol Choi

Andrew Cohen

Oleh Dzyubachyk

Adrian Galdran

Tianyuan Gan

Tianqi Guo

Pradyumna Gupta

M. Haithami

Edward Ho

Ikbeom Jang

Zhili Li

Zheng Luo

Filip Lux

Sokratis Makrogiannis

Dominikus Muller

Young-Tack Oh

Subeen Pang

Constantin Pape

Gorkem Polat

Charlotte Rosalie Reed

Kanghyun Ryu

Tim Scherr

Vajira L. Thambawita

Haoyu Wang

Xinliang Wang

Kele Xu

H.-I. Yeh

Doyeob Yeo

Yi Yuan

Yan Zeng

Xingwen Zhao

Julian Ronald Abbing

Jannes Adam

Nagesh Adluru

Niklas Agethen

S. Ahmed

Yasmina Al Khalil

Mireia Alenya

Esa J. Alhoniemi

C. An

Talha E Anwar

Tewodros Arega

Netanell Avisdris

D. Aydogan

Yi-Shi Bai

Maria Baldeon Calisto

Berke Doga Basaran

Marcel Beetz

Cheng Bian

Hao-xuan Bian

Kevin Blansit

Louise Bloch

Robert Bohnsack

Sara Bosticardo

J. Breen

Mikael Brudfors

Raphael Brungel

Mariano Cabezas

Alberto Cacciola

Zhiwei Chen

Yucong Chen

Dan Chen

Minjeong Cho

Min-Kook Choi

Chuantao Xie Chuantao Xie

Dana Cobzas

Jorge Corral Acero

Sujit Kumar Das

Marcela de Oliveira

Hanqiu Deng

Guiming Dong

Lars Doorenbos

Cory Efird

Di Fan

Mehdi Fatan Serj

Alexandre Fenneteau

Lucas Fidon

Patryk Filipiak

Ren'e Finzel

Nuno Renato Freitas

C. Friedrich

Mitchell J. Fulton

Finn Gaida

Francesco Galati

Christoforos Galazis

Changna Gan

Zheyao Gao

Sheng Gao

Matej Gazda

Beerend G. A. Gerats

Neil Getty

Adam Gibicar

Ryan J. Gifford

Sajan Gohil

Maria Grammatikopoulou

Daniel Grzech

Orhun Guley

Timo Gunnemann

Chun-Hai Guo

Sylvain Guy

Heonjin Ha

Luyi Han

Ilseok Han

Ali Hatamizadeh

Tianhai He

Ji-Wu Heo

Sebastian Hitziger

SeulGi Hong

Seungbum Hong

Rian Huang

Zi-You Huang

Markus Huellebrand

Stephan Huschauer

M. Hussain

Tomoo Inubushi

Ece Isik Polat

Mojtaba Jafaritadi

Seonghun Jeong

Bailiang Jian

Yu Jiang

Zhifan Jiang

Yu Jin

Smriti Joshi

A. Kadkhodamohammadi

R. A. Kamraoui

Inhak Kang

Jun-Su Kang

Davood Karimi

April Ellahe Khademi

Muhammad Irfan Khan

Suleiman A. Khan

Rishab Khantwal

Kwang-Ju Kim

Timothy Lee Kline

Satoshi Kondo

Elina Kontio

Adrian Krenzer

Artem Kroviakov

Hugo J. Kuijf

Satyadwyoom Kumar

Francesco La Rosa

Abhishek Lad

Doohee Lee

Minho Lee

Chiara Lena

Hao Li

Ling Li

Xingyu Li

F. Liao

Kuan-Ya Liao

Arlindo L. Oliveira

Chaonan Lin

Shanhai Lin

Akis Linardos

M. Linguraru

Han Liu

Tao Liu

Dian Liu

Yanling Liu

Joao Lourencco-Silva

Jing Lu

Jia Lu

Imanol Luengo

Christina Bach Lund

Huan Minh Luu

Yingqi Lv

Leon Maechler

L. SinaMansour

Kenji Marshall

Moona Mazher

Richard McKinley

Alfonso Medela

Felix Meissen

Mingyuan Meng

Dylan Bradley Miller

S. Mirjahanmardi

Arnab Kumar Mishra

Samir Mitha

Hassan Mohy-ud-Din

Tony C. W. Mok

Gowtham Krishnan Murugesan

Enamundram Naga Karthik

Sahil Nalawade

Jakub Nalepa

M. Naser

Ramin Nateghi

Hammad Naveed

Quang-Minh Nguyen

Cuong Nguyen Quoc

Bruno Oliveira

David Owen

Jimut Bahan Pal

Junwen Pan

W. Pan

Winnie Pang

Bogyu Park

Vivek G. Pawar

K. Pawar

Michael Peven

Lena Philipp

Tomasz Pieciak

Szymon S Płotka

Marcel Plutat

Fattane Pourakpour

Domen Prelovznik

K. Punithakumar

Abdul Qayyum

Sandro Queir'os

Arman Rahmim

Salar Razavi

Jintao Ren

Mina Rezaei

Jonathan Adam Rico

ZunHyan Rieu

Markus Rink

Johannes Roth

Yusely Ruiz-gonzalez

Numan Saeed

Anindo Saha

Mostafa M. Sami Salem

Ricardo Sanchez-matilla

Kurt G Schilling

Weizhen Shao

Zhiqiang Shen

Ruize Shi

Pengcheng Shi

Daniel Sobotka

Th'eodore Soulier

Bella Specktor Fadida

D. Stoyanov

Timothy Sum Hon Mun

Xiao-Fu Sun

Rong Tao

Franz Thaler

Antoine Th'eberge

Felix Thielke

Helena R. Torres

K. Wahid

Jiacheng Wang

Yifei Wang

W. Wang

Xiong Jun Wang

Jianhui Wen

Ning Wen

Marek Wodziński

Yehong Wu

Fangfang Xia

Tianqi Xiang

Cheng Xiaofei

Lizhang Xu

Tingting Xue

Yu‐Xia Yang

Lingxian Yang

Kai Yao

Huifeng Yao

Amirsaeed Yazdani

Michael Yip

Hwa-Seong Yoo

Fereshteh Yousefirizi

Shu-Fen Yu

Lei Yu

Jonathan Zamora

Ramy Ashraf Zeineldin

Dewen Zeng

Jianpeng Zhang

Bokai Zhang

Jiapeng Zhang

Fangxi Zhang

Huahong Zhang

Zhongchen Zhao

Zixuan Zhao

Jia Zhao

Can Zhao

Q. Zheng

Yuheng Zhi

Ziqi Zhou

Baosheng Zou

Klaus Maier-Hein

PAUL F. JÄGER

Annette Kopp-Schneider

Lena Maier-Hein

The number of international benchmarking competitions is steadily increasing in various fields of machine learning (ML) research and practic… (see more)e. So far, however, little is known about the common practice as well as bottlenecks faced by the community in tackling the research questions posed. To shed light on the status quo of algorithm development in the specific field of biomedical imaging analysis, we designed an international survey that was issued to all participants of challenges conducted in conjunction with the IEEE ISBI 2021 and MICCAI 2021 conferences (80 competitions in total). The survey covered participants' expertise and working environments, their chosen strategies, as well as algorithm characteristics. A median of 72% challenge participants took part in the survey. According to our results, knowledge exchange was the primary incentive (70%) for participation, while the reception of prize money played only a minor role (16%). While a median of 80 working hours was spent on method development, a large portion of participants stated that they did not have enough time for method development (32%). 25% perceived the infrastructure to be a bottleneck. Overall, 94% of all solutions were deep learning-based. Of these, 84% were based on standard architectures. 43% of the respondents reported that the data samples (e.g., images) were too large to be processed at once. This was most commonly addressed by patch-based training (69%), downsampling (37%), and solving 3D analysis tasks as a series of 2D tasks. K-fold cross-validation on the training set was performed by only 37% of the participants and only 50% of the participants performed ensembling based on multiple identical models (61%) or heterogeneous models (39%). 48% of the respondents applied postprocessing steps.

2022-12-16

ArXiv (preprint)

Biomedical image analysis competitions: The state of current participation practice

Matthias Eisenmann

Annika Reinke

Vivienn Weru

Minu Dietlinde Tizabi

Fabian Isensee

T. Adler

PATRICK GODAU

Veronika Cheplygina

Michal Kozubek

Sharib Ali

Anubha Gupta

Jan. Kybic

Alison Professor Noble

Carlos Ortiz de Sol'orzano

Samiksha Pachade

Caroline Petitjean

Daniel Sage

Donglai Wei

Elizabeth Wilden

Deepak Alapatt … (see 334 more)

Vincent Andrearczyk

Ujjwal Baid

Spyridon Bakas

Niranjan Balu

Sophia Bano

Vivek Singh Bawa

Jorge Bernal

Sebastian Bodenstedt

Alessandro Casella

Jinwook Choi

Olivier Commowick

M. Daum

Adrien Depeursinge

Reuben Dorent

J. Egger

H. Eichhorn

Sandy Engelhardt

Melanie Ganz

Gabriel Girard

Lasse Donovan Hansen

Mattias Paul Heinrich

Nicholas Heller

Alessa Hering

Arnaud Huaulm'e

Hyunjeong Kim

Bennett Landman

Hongwei Bran Li

Jianning Li

Junfang Ma

Anne L. Martel

Carlos Mart'in-Isla

Bjoern Menze

Chinedu Innocent Nwoye

Valentin Oreiller

Nicolas Padoy

Sarthak Pati

Kelly Payette

Carole H. Sudre

K. V. Wijnen

Armine Vardazaryan

Tom Kamiel Magda Vercauteren

Martin Wagner

Chuanbo Wang

Moi Hoon Yap

Zeyun Yu

Chuner Yuan

Maximilian Zenk

Aneeq Zia

David Zimmerer

Rina Bao

Chanyeol Choi

Andrew Cohen

Oleh Dzyubachyk

Adrian Galdran

Tianyuan Gan

Tianqi Guo

Pradyumna Gupta

M. Haithami

Edward Ho

Ikbeom Jang

Zhili Li

Zheng Luo

Filip Lux

Sokratis Makrogiannis

Dominikus Muller

Young-Tack Oh

Subeen Pang

Constantin Pape

Gorkem Polat

Charlotte Rosalie Reed

Kanghyun Ryu

Tim Scherr

Vajira L. Thambawita

Haoyu Wang

Xinliang Wang

Kele Xu

H.-I. Yeh

Doyeob Yeo

Yi Yuan

Yan Zeng

Xingwen Zhao

Julian Ronald Abbing

Jannes Adam

Nagesh Adluru

Niklas Agethen

S. Ahmed

Yasmina Al Khalil

Mireia Alenya

Esa J. Alhoniemi

C. An

Talha E Anwar

Tewodros Arega

Netanell Avisdris

D. Aydogan

Yi-Shi Bai

Maria Baldeon Calisto

Berke Doga Basaran

Marcel Beetz

Cheng Bian

Hao-xuan Bian

Kevin Blansit

Louise Bloch

Robert Bohnsack

Sara Bosticardo

J. Breen

Mikael Brudfors

Raphael Brungel

Mariano Cabezas

Alberto Cacciola

Zhiwei Chen

Yucong Chen

Dan Chen

Minjeong Cho

Min-Kook Choi

Chuantao Xie Chuantao Xie

Dana Cobzas

Jorge Corral Acero

Sujit Kumar Das

Marcela de Oliveira

Hanqiu Deng

Guiming Dong

Lars Doorenbos

Cory Efird

Di Fan

Mehdi Fatan Serj

Alexandre Fenneteau

Lucas Fidon

Patryk Filipiak

Ren'e Finzel

Nuno Renato Freitas

C. Friedrich

Mitchell J. Fulton

Finn Gaida

Francesco Galati

Christoforos Galazis

Changna Gan

Zheyao Gao

Sheng Gao

Matej Gazda

Beerend G. A. Gerats

Neil Getty

Adam Gibicar

Ryan J. Gifford

Sajan Gohil

Maria Grammatikopoulou

Daniel Grzech

Orhun Guley

Timo Gunnemann

Chun-Hai Guo

Sylvain Guy

Heonjin Ha

Luyi Han

Ilseok Han

Ali Hatamizadeh

Tianhai He

Ji-Wu Heo

Sebastian Hitziger

SeulGi Hong

Seungbum Hong

Rian Huang

Zi-You Huang

Markus Huellebrand

Stephan Huschauer

M. Hussain

Tomoo Inubushi

Ece Isik Polat

Mojtaba Jafaritadi

Seonghun Jeong

Bailiang Jian

Yu Jiang

Zhifan Jiang

Yu Jin

Smriti Joshi

A. Kadkhodamohammadi

R. A. Kamraoui

Inhak Kang

Jun-Su Kang

Davood Karimi

April Ellahe Khademi

Muhammad Irfan Khan

Suleiman A. Khan

Rishab Khantwal

Kwang-Ju Kim

Timothy Lee Kline

Satoshi Kondo

Elina Kontio

Adrian Krenzer

Artem Kroviakov

Hugo J. Kuijf

Satyadwyoom Kumar

Francesco La Rosa

Abhishek Lad

Doohee Lee

Minho Lee

Chiara Lena

Hao Li

Ling Li

Xingyu Li

F. Liao

Kuan-Ya Liao

Arlindo L. Oliveira

Chaonan Lin

Shanhai Lin

Akis Linardos

M. Linguraru

Han Liu

Tao Liu

Dian Liu

Yanling Liu

Joao Lourencco-Silva

Jing Lu

Jia Lu

Imanol Luengo

Christina Bach Lund

Huan Minh Luu

Yingqi Lv

Leon Maechler

L. SinaMansour

Kenji Marshall

Moona Mazher

Richard McKinley

Alfonso Medela

Felix Meissen

Mingyuan Meng

Dylan Bradley Miller

S. Mirjahanmardi

Arnab Kumar Mishra

Samir Mitha

Hassan Mohy-ud-Din

Tony C. W. Mok

Gowtham Krishnan Murugesan

Enamundram Naga Karthik

Sahil Nalawade

Jakub Nalepa

M. Naser

Ramin Nateghi

Hammad Naveed

Quang-Minh Nguyen

Cuong Nguyen Quoc

Bruno Oliveira

David Owen

Jimut Bahan Pal

Junwen Pan

W. Pan

Winnie Pang

Bogyu Park

Vivek G. Pawar

K. Pawar

Michael Peven

Lena Philipp

Tomasz Pieciak

Szymon S Płotka

Marcel Plutat

Fattane Pourakpour

Domen Prelovznik

K. Punithakumar

Abdul Qayyum

Sandro Queir'os

Arman Rahmim

Salar Razavi

Jintao Ren

Mina Rezaei

Jonathan Adam Rico

ZunHyan Rieu

Markus Rink

Johannes Roth

Yusely Ruiz-gonzalez

Numan Saeed

Anindo Saha

Mostafa M. Sami Salem

Ricardo Sanchez-matilla

Kurt G Schilling

Weizhen Shao

Zhiqiang Shen

Ruize Shi

Pengcheng Shi

Daniel Sobotka

Th'eodore Soulier

Bella Specktor Fadida

D. Stoyanov

Timothy Sum Hon Mun

Xiao-Fu Sun

Rong Tao

Franz Thaler

Antoine Th'eberge

Felix Thielke

Helena R. Torres

K. Wahid

Jiacheng Wang

Yifei Wang

W. Wang

Xiong Jun Wang

Jianhui Wen

Ning Wen

Marek Wodziński

Yehong Wu

Fangfang Xia

Tianqi Xiang

Cheng Xiaofei

Lizhang Xu

Tingting Xue

Yu‐Xia Yang

Lingxian Yang

Kai Yao

Huifeng Yao

Amirsaeed Yazdani

Michael Yip

Hwa-Seong Yoo

Fereshteh Yousefirizi

Shu-Fen Yu

Lei Yu

Jonathan Zamora

Ramy Ashraf Zeineldin

Dewen Zeng

Jianpeng Zhang

Bokai Zhang

Jiapeng Zhang

Fangxi Zhang

Huahong Zhang

Zhongchen Zhao

Zixuan Zhao

Jia Zhao

Can Zhao

Q. Zheng

Yuheng Zhi

Ziqi Zhou

Baosheng Zou

Klaus Maier-Hein

PAUL F. JÄGER

Annette Kopp-Schneider

Lena Maier-Hein

The number of international benchmarking competitions is steadily increasing in various fields of machine learning (ML) research and practic… (see more)e. So far, however, little is known about the common practice as well as bottlenecks faced by the community in tackling the research questions posed. To shed light on the status quo of algorithm development in the specific field of biomedical imaging analysis, we designed an international survey that was issued to all participants of challenges conducted in conjunction with the IEEE ISBI 2021 and MICCAI 2021 conferences (80 competitions in total). The survey covered participants' expertise and working environments, their chosen strategies, as well as algorithm characteristics. A median of 72% challenge participants took part in the survey. According to our results, knowledge exchange was the primary incentive (70%) for participation, while the reception of prize money played only a minor role (16%). While a median of 80 working hours was spent on method development, a large portion of participants stated that they did not have enough time for method development (32%). 25% perceived the infrastructure to be a bottleneck. Overall, 94% of all solutions were deep learning-based. Of these, 84% were based on standard architectures. 43% of the respondents reported that the data samples (e.g., images) were too large to be processed at once. This was most commonly addressed by patch-based training (69%), downsampling (37%), and solving 3D analysis tasks as a series of 2D tasks. K-fold cross-validation on the training set was performed by only 37% of the participants and only 50% of the participants performed ensembling based on multiple identical models (61%) or heterogeneous models (39%). 48% of the respondents applied postprocessing steps.

2022-12-16

ArXiv (preprint)

Biomedical image analysis competitions: The state of current participation practice

Matthias Eisenmann

Annika Reinke

Vivienn Weru

Minu Dietlinde Tizabi

Fabian Isensee

T. Adler

PATRICK GODAU

Veronika Cheplygina

Michal Kozubek

Sharib Ali

Anubha Gupta

Jan. Kybic

Alison Professor Noble

Carlos Ortiz de Sol'orzano

Samiksha Pachade

Caroline Petitjean

Daniel Sage

Donglai Wei

Elizabeth Wilden

Deepak Alapatt … (see 334 more)

Vincent Andrearczyk

Ujjwal Baid

Spyridon Bakas

Niranjan Balu

Sophia Bano

Vivek Singh Bawa

Jorge Bernal

Sebastian Bodenstedt

Alessandro Casella

Jinwook Choi

Olivier Commowick

M. Daum

Adrien Depeursinge

Reuben Dorent

J. Egger

H. Eichhorn

Sandy Engelhardt

Melanie Ganz

Gabriel Girard

Lasse Donovan Hansen

Mattias Paul Heinrich

Nicholas Heller

Alessa Hering

Arnaud Huaulm'e

Hyunjeong Kim

Bennett Landman

Hongwei Bran Li

Jianning Li

Junfang Ma

Anne L. Martel

Carlos Mart'in-Isla

Bjoern Menze

Chinedu Innocent Nwoye

Valentin Oreiller

Nicolas Padoy

Sarthak Pati

Kelly Payette

Carole H. Sudre

K. V. Wijnen

Armine Vardazaryan

Tom Kamiel Magda Vercauteren

Martin Wagner

Chuanbo Wang

Moi Hoon Yap

Zeyun Yu

Chuner Yuan

Maximilian Zenk

Aneeq Zia

David Zimmerer

Rina Bao

Chanyeol Choi

Andrew Cohen

Oleh Dzyubachyk

Adrian Galdran

Tianyuan Gan

Tianqi Guo

Pradyumna Gupta

M. Haithami

Edward Ho

Ikbeom Jang

Zhili Li

Zheng Luo

Filip Lux

Sokratis Makrogiannis

Dominikus Muller

Young-Tack Oh

Subeen Pang

Constantin Pape

Gorkem Polat

Charlotte Rosalie Reed

Kanghyun Ryu

Tim Scherr

Vajira L. Thambawita

Haoyu Wang

Xinliang Wang

Kele Xu

H.-I. Yeh

Doyeob Yeo

Yi Yuan

Yan Zeng

Xingwen Zhao

Julian Ronald Abbing

Jannes Adam

Nagesh Adluru

Niklas Agethen

S. Ahmed

Yasmina Al Khalil

Mireia Alenya

Esa J. Alhoniemi

C. An

Talha E Anwar

Tewodros Arega

Netanell Avisdris

D. Aydogan

Yi-Shi Bai

Maria Baldeon Calisto

Berke Doga Basaran

Marcel Beetz

Cheng Bian

Hao-xuan Bian

Kevin Blansit

Louise Bloch

Robert Bohnsack

Sara Bosticardo

J. Breen

Mikael Brudfors

Raphael Brungel

Mariano Cabezas

Alberto Cacciola

Zhiwei Chen

Yucong Chen

Dan Chen

Minjeong Cho

Min-Kook Choi

Chuantao Xie Chuantao Xie

Dana Cobzas

Jorge Corral Acero

Sujit Kumar Das

Marcela de Oliveira

Hanqiu Deng

Guiming Dong

Lars Doorenbos

Cory Efird

Di Fan

Mehdi Fatan Serj

Alexandre Fenneteau

Lucas Fidon

Patryk Filipiak

Ren'e Finzel

Nuno Renato Freitas

C. Friedrich

Mitchell J. Fulton

Finn Gaida

Francesco Galati

Christoforos Galazis

Changna Gan

Zheyao Gao

Sheng Gao

Matej Gazda

Beerend G. A. Gerats

Neil Getty

Adam Gibicar

Ryan J. Gifford

Sajan Gohil

Maria Grammatikopoulou

Daniel Grzech

Orhun Guley

Timo Gunnemann

Chun-Hai Guo

Sylvain Guy

Heonjin Ha

Luyi Han

Ilseok Han

Ali Hatamizadeh

Tianhai He

Ji-Wu Heo

Sebastian Hitziger

SeulGi Hong

Seungbum Hong

Rian Huang

Zi-You Huang

Markus Huellebrand

Stephan Huschauer

M. Hussain

Tomoo Inubushi

Ece Isik Polat

Mojtaba Jafaritadi

Seonghun Jeong

Bailiang Jian

Yu Jiang

Zhifan Jiang

Yu Jin

Smriti Joshi

A. Kadkhodamohammadi

R. A. Kamraoui

Inhak Kang

Jun-Su Kang

Davood Karimi

April Ellahe Khademi

Muhammad Irfan Khan

Suleiman A. Khan

Rishab Khantwal

Kwang-Ju Kim

Timothy Lee Kline

Satoshi Kondo

Elina Kontio

Adrian Krenzer

Artem Kroviakov

Hugo J. Kuijf

Satyadwyoom Kumar

Francesco La Rosa

Abhishek Lad

Doohee Lee

Minho Lee

Chiara Lena

Hao Li

Ling Li

Xingyu Li

F. Liao

Kuan-Ya Liao

Arlindo L. Oliveira

Chaonan Lin

Shanhai Lin

Akis Linardos

M. Linguraru

Han Liu

Tao Liu

Dian Liu

Yanling Liu

Joao Lourencco-Silva

Jing Lu

Jia Lu

Imanol Luengo

Christina Bach Lund

Huan Minh Luu

Yingqi Lv

Leon Maechler

L. SinaMansour

Kenji Marshall

Moona Mazher

Richard McKinley

Alfonso Medela

Felix Meissen

Mingyuan Meng

Dylan Bradley Miller

S. Mirjahanmardi

Arnab Kumar Mishra

Samir Mitha

Hassan Mohy-ud-Din

Tony C. W. Mok

Gowtham Krishnan Murugesan

Enamundram Naga Karthik

Sahil Nalawade

Jakub Nalepa

M. Naser

Ramin Nateghi

Hammad Naveed

Quang-Minh Nguyen

Cuong Nguyen Quoc

Bruno Oliveira

David Owen

Jimut Bahan Pal

Junwen Pan

Wei-Dong Pan

Winnie Pang

Bogyu Park

Vivek G. Pawar

Kamlesh Pawar

Michael Peven

Lena Philipp

Tomasz Pieciak

Szymon S Płotka

Marcel Plutat

Fattane Pourakpour

Domen Prelovznik

K. Punithakumar

Abdul Qayyum

Sandro Queir'os

Arman Rahmim

Salar Razavi

Jintao Ren

Mina Rezaei

Jonathan Adam Rico

ZunHyan Rieu

Markus Rink

Johannes Roth

Yusely Ruiz-gonzalez

Numan Saeed

Anindo Saha

Mostafa M. Sami Salem

Ricardo Sanchez-matilla

Kurt G Schilling

Weizhen Shao

Zhiqiang Shen

Ruize Shi

Pengcheng Shi

Daniel Sobotka

Th'eodore Soulier

Bella Specktor Fadida

D. Stoyanov

Timothy Sum Hon Mun

Xiao-Fu Sun

Rong Tao

Franz Thaler

Antoine Th'eberge

Felix Thielke

Helena R. Torres

K. Wahid

Jiacheng Wang

Yifei Wang

Wei David Wang

Xiong Jun Wang

Jianhui Wen

Ning Wen

Marek Wodziński

Yehong Wu

Fangfang Xia

Tianqi Xiang

Cheng Xiaofei

Lizhang Xu

Tingting Xue

Yu‐Xia Yang

Lingxian Yang

Kai Yao

Huifeng Yao

Amirsaeed Yazdani

Michael Yip

Hwa-Seong Yoo

Fereshteh Yousefirizi

Shu-Fen Yu

Lei Yu

Jonathan Zamora

Ramy A. Zeineldin

Dewen Zeng

Jianpeng Zhang

Bokai Zhang

Jiapeng Zhang

Fangxi Zhang

Huahong Zhang

Zhongchen Zhao

Zixuan Zhao

Jia Zhao

Can Zhao

Qiuyue Zheng

Yuheng Zhi

Ziqi Zhou

Baosheng Zou

Klaus Maier-Hein

PAUL F. JÄGER

Annette Kopp-Schneider

Lena Maier-Hein

2022-12-16

ArXiv (preprint)

Personalized Prediction of Future Lesion Activity and Treatment Effect in Multiple Sclerosis from Baseline MRI

Joshua D. Durso-Finley

Jean-Pierre R. Falet

Douglas Arnold

Precision medicine for chronic diseases such as multiple sclerosis (MS) involves choosing a treatment which best balances efficacy and side … (see more)effects/preferences for individual patients. Making this choice as early as possible is important, as delays in finding an effective therapy can lead to irreversible disability accrual. To this end, we present the first deep neural network model for individualized treatment decisions from baseline magnetic resonance imaging (MRI) (with clinical information if available) for MS patients which (a) predicts future new and enlarging T2 weighted (NE-T2) lesion counts on follow-up MRI on multiple treatments and (b) estimates the conditional average treatment effect (CATE), as defined by the predicted future suppression of NE-T2 lesions, between different treatment options relative to placebo. Our model is validated on a proprietary federated dataset of 1817 multi-sequence MRIs acquired from MS patients during four multi-centre randomized clinical trials. Our framework achieves high average precision in the binarized regression of future NE-T2 lesions on five different treatments, identifies heterogeneous treatment effects, and provides a personalized treatment recommendation that accounts for treatment-associated risk (side effects, patient preference, administration difficulties,...).

2022-12-04

Proceedings of The 5th International Conference on Medical Imaging with Deep Learning (published)

openreview.net

Clinically Plausible Pathology-Anatomy Disentanglement in Patient Brain MRI with Structured Variational Priors

Anjun Hu

Jean-Pierre R. Falet

Changjian Shui

Douglas Arnold

Sotirios A. Tsaftaris

We propose a hierarchically structured variational inference model for accurately disentangling observable evidence of disease (e.g. brain l… (see more)esions or atrophy) from subject-specific anatomy in brain MRIs. With flexible, partially autoregressive priors, our model (1) addresses the subtle and fine-grained dependencies that typically exist between anatomical and pathological generating factors of an MRI to ensure the clinical validity of generated samples; (2) preserves and disentangles finer pathological details pertaining to a patient's disease state. Additionally, we experiment with an alternative training configuration where we provide supervision to a subset of latent units. It is shown that (1) a partially supervised latent space achieves a higher degree of disentanglement between evidence of disease and subject-specific anatomy; (2) when the prior is formulated with an autoregressive structure, knowledge from the supervision can propagate to the unsupervised latent units, resulting in more informative latent representations capable of modelling anatomy-pathology interdependencies.

2022-11-15

ArXiv (preprint)

Rethinking Generalization: The Impact of Annotation Style on Medical Image Segmentation

Jillian L. Cardinell

Justin Szeto

Raghav Mehta

Jean-Pierre R. Falet

Douglas Arnold

Sotirios A. Tsaftaris

Generalization is an important attribute of machine learning models, particularly for those that are to be deployed in a medical context, wh… (see more)ere unreliable predictions can have real world consequences. While the failure of models to generalize across datasets is typically attributed to a mismatch in the data distributions, performance gaps are often a consequence of biases in the "ground-truth" label annotations. This is particularly important in the context of medical image segmentation of pathological structures (e.g. lesions), where the annotation process is much more subjective, and affected by a number underlying factors, including the annotation protocol, rater education/experience, and clinical aims, among others. In this paper, we show that modeling annotation biases, rather than ignoring them, poses a promising way of accounting for differences in annotation style across datasets. To this end, we propose a generalized conditioning framework to (1) learn and account for different annotation styles across multiple datasets using a single model, (2) identify similar annotation styles across different datasets in order to permit their effective aggregation, and (3) fine-tune a fully trained model to a new annotation style with just a few samples. Next, we present an image-conditioning approach to model annotation styles that correlate with specific image features, potentially enabling detection biases to be more easily identified.

2022-10-31

ArXiv (preprint)

Estimating individual treatment effect on disability progression in multiple sclerosis using deep learning

Jean-Pierre R. Falet

Joshua D. Durso-Finley

Julien Schroeter

Francesca Bovis

Maria-Pia Sormani

Doina Precup

Douglas Arnold

2022-09-26

Nature Communications (published)

Counterfactual Image Synthesis for Discovery of Personalized Predictive Image Markers

Amar Kumar

Anjun Hu

Jean-Pierre R. Falet

Douglas Arnold

Sotirios A. Tsaftaris

2022-08-03

ArXiv (preprint)

Information Gain Sampling for Active Learning in Medical Image Classification

Raghav Mehta

Changjian Shui

2022-08-01

ArXiv (preprint)

GP.2 Deep learning prediction of response to disease modifying therapy in primary progressive multiple sclerosis

JR Falet

Joshua D. Durso-Finley

Julien Schroeter

Francesca Bovis

M Sormani

D Precup

DL Arnold

Background: Only one disease modifying therapy (DMT), ocrelizumab, was found to slow disability progression in primary progressive multiple … (see more)sclerosis (PPMS). Modeling the conditional average treatment effect (CATE) using deep learning could identify individuals more responsive to DMTs, allowing for predictive enrichment to increase the power of future clinical trials. Methods: Baseline clinical and MRI data were acquired as part of three placebo-controlled randomized clinical trials: ORATORIO (ocrelizumab), OLYMPUS (rituximab) and ARPEGGIO (laquinimod). Data from ORATORIO and OLYMPUS was separated into a training (70%) and testing (30%) set, while ARPEGGIO served as additional validation. An ensemble of multitask multilayer perceptrons was trained to predict the rate of disability progression on both treatment and placebo to estimate CATE. Results: The model could separate individuals based on their predicted treatment effect. The top 25% of individuals predicted to respond most have a larger effect size (HR 0.442, p=0.0497) than the entire group (HR 0.787, p=0.292). The model could also identify responders to laquinimod. A simulated study where the 50% most responsive individuals are randomized would require 6-times less participants to detect a significant effect. Conclusions: Individuals with PPMS who respond favourably to DMTs can be identified using deep learning based on their baseline clinical and imaging characteristics.

2022-06-24

Canadian Journal of Neurological Sciences / Journal Canadien des Sciences Neurologiques (published)

Metrics Reloaded - A new recommendation framework for biomedical image analysis validation

Annika Reinke

Lena Maier-Hein

Evangelia Christodoulou

Ben Glocker

Patrick Scholz

Fabian Isensee

Jens Kleesiek

Michal Kozubek

Mauricio Reyes

Michael Alexander Riegler

Manuel Wiesenfarth

Michael Baumgartner

Matthias Eisenmann

DOREEN HECKMANN-NÖTZEL

Ali Emre Kavur

TIM RÄDSCH

Minu D. Tizabi

LAURA ACION

Michela Antonelli

Tal Arbel … (see 48 more)

Spyridon Bakas

Peter Bankhead

Arriel Benis

M. Jorge Cardoso

Veronika Cheplygina

Beth A Cimini

Gary S. Collins

Keyvan Farahani

Bram van Ginneken

Fred A Hamprecht

Daniel A. Hashimoto

Michael M. Hoffman

Merel Huisman

Pierre Jannin

Charles Kahn

Alexandros Karargyris

Alan Karthikesalingam

Hannes Kenngott

Annette Kopp-Schneider

Anna Kreshuk

Tahsin Kurc

Bennett Landman

GEERT LITJENS

Amin Madani

Klaus Maier-Hein

Anne Martel

Peter Mattson

ERIK MEIJERING

Bjoern Menze

David Moher

KAREL G.M. MOONS

Henning Müller