Portrait de Brennan Nichyporuk n'est pas disponible

Brennan Nichyporuk

nichypob@mila.quebec
Scientifique de recherche, Innovation, développement et technologies

Publications

Optimizing Operating Points for High Performance Lesion Detection and Segmentation Using Lesion Size Reweighting
Brennan Nichyporuk
Justin Szeto
Douglas Arnold
Tal Arbel
There are many clinical contexts which require accurate detection and segmentation of all focal pathologies (e.g. lesions, tumours) in patie… (voir plus)nt images. In cases where there are a mix of small and large lesions, standard binary cross entropy loss will result in better segmentation of large lesions at the expense of missing small ones. Adjusting the operating point to accurately detect all lesions generally leads to oversegmentation of large lesions. In this work, we propose a novel reweighing strategy to eliminate this performance gap, increasing small pathology detection performance while maintaining segmentation accuracy. We show that our reweighing strategy vastly outperforms competing strategies based on experiments on a large scale, multi-scanner, multi-center dataset of Multiple Sclerosis patient images.
2021-05-10
MIDL.io/2021/Conference/Short (poster)
openreview.net
openreview.net
Accounting for Variance in Machine Learning Benchmarks
Xavier Bouthillier
Pierre Delaunay
Mirko Bronzi
Assya Trofimov
Brennan Nichyporuk
Justin Szeto
Naz Sepah
Edward Raff
Kanika Madan
Vikram Voleti
Samira Ebrahimi Kahou
Vincent Michalski
Dmitriy Serdyuk
Tal Arbel
Chris Pal
Gael Varoquaux
Pascal Vincent
Strong empirical evidence that one machine-learning algorithm A outperforms another one B ideally calls for multiple trials optimizing the l… (voir plus)earning pipeline over sources of variation such as data sampling, data augmentation, parameter initialization, and hyperparameters choices. This is prohibitively expensive, and corners are cut to reach conclusions. We model the whole benchmarking process, revealing that variance due to data sampling, parameter initialization and hyperparameter choice impact markedly the results. We analyze the predominant comparison methods used today in the light of this variance. We show a counter-intuitive result that adding more sources of variation to an imperfect estimator approaches better the ideal estimator at a 51 times reduction in compute cost. Building on these results, we study the error rate of detecting improvements, on five different deep-learning tasks/architectures. This study leads us to propose recommendations for performance comparisons.
2021-04-05
Conference on Machine Learning and Systems (Accept)
doi.org
arxiv.org