Publications

Where Off-Policy Deep Reinforcement Learning Fails
Scott Fujimoto
David Meger
Doina Precup
This work examines batch reinforcement learning–the task of maximally exploiting a given batch of off-policy data, without further data co… (see more)llection. We demonstrate that due to errors introduced by extrapolation, standard off-policy deep reinforcement learning algorithms, such as DQN and DDPG, are only capable of learning with data correlated to their current policy, making them ineffective for most off-policy applications. We introduce a novel class of off-policy algorithms, batch-constrained reinforcement learning, which restricts the action space to force the agent towards behaving on-policy with respect to a subset of the given data. We extend this notion to deep reinforcement learning, and to the best of our knowledge, present the first continuous control deep reinforcement learning algorithm which can learn effectively from uncorrelated off-policy data.
2018-09-26
(published)
openreview.net
openreview.net
Width of Minima Reached by Stochastic Gradient Descent is Influenced by Learning Rate to Batch Size Ratio
Stanisław Jastrzębski
Zac Kenton
Devansh Arpit
Nicolas Ballas
Asja Fischer
Yoshua Bengio
Amos Storkey
2018-09-26
Artificial Neural Networks and Machine Learning – ICANN 2018 (published)
doi.org
Exploring Uncertainty Measures in Deep Networks for Multiple Sclerosis Lesion Detection and Segmentation
Tanya Nair
Doina Precup
Douglas L. Arnold
Tal Arbel
Deep learning (DL) networks have recently been shown to outperform other segmentation methods on various public, medical-image challenge dat… (see more)asets [3,11,16], especially for large pathologies. However, in the context of diseases such as Multiple Sclerosis (MS), monitoring all the focal lesions visible on MRI sequences, even very small ones, is essential for disease staging, prognosis, and evaluating treatment efficacy. Moreover, producing deterministic outputs hinders DL adoption into clinical routines. Uncertainty estimates for the predictions would permit subsequent revision by clinicians. We present the first exploration of multiple uncertainty estimates based on Monte Carlo (MC) dropout [4] in the context of deep networks for lesion detection and segmentation in medical images. Specifically, we develop a 3D MS lesion segmentation CNN, augmented to provide four different voxel-based uncertainty measures based on MC dropout. We train the network on a proprietary, large-scale, multi-site, multi-scanner, clinical MS dataset, and compute lesion-wise uncertainties by accumulating evidence from voxel-wise uncertainties within detected lesions. We analyze the performance of voxel-based segmentation and lesion-level detection by choosing operating points based on the uncertainty. Empirical evidence suggests that uncertainty measures consistently allow us to choose superior operating points compared only using the network's sigmoid output as a probability.
2018-09-25
Medical Image Computing and Computer Assisted Intervention – MICCAI 2018 (published)
doi.org
arxiv.org
How can deep learning advance computational modeling of sensory information processing?
Jessica A.F. Thompson
Yoshua Bengio
Elia Formisano
Marc Schönwiesner
Deep learning, computational neuroscience, and cognitive science have overlapping goals related to understanding intelligence such that perc… (see more)eption and behaviour can be simulated in computational systems. In neuroimaging, machine learning methods have been used to test computational models of sensory information processing. Recently, these model comparison techniques have been used to evaluate deep neural networks (DNNs) as models of sensory information processing. However, the interpretation of such model evaluations is muddied by imprecise statistical conclusions. Here, we make explicit the types of conclusions that can be drawn from these existing model comparison techniques and how these conclusions change when the model in question is a DNN. We discuss how DNNs are amenable to new model comparison techniques that allow for stronger conclusions to be made about the computational mechanisms underlying sensory information processing.
2018-09-24
ArXiv (preprint)
arxiv.org
arxiv.org
On the Learning Dynamics of Deep Neural Networks
Remi Tachet des Combes
Mohammad Pezeshki
Samira Shabanian
Aaron Courville
Yoshua Bengio
While a lot of progress has been made in recent years, the dynamics of learning in deep nonlinear neural networks remain to this day largely… (see more) misunderstood. In this work, we study the case of binary classification and prove various properties of learning in such networks under strong assumptions such as linear separability of the data. Extending existing results from the linear case, we confirm empirical observations by proving that the classification error also follows a sigmoidal shape in nonlinear architectures. We show that given proper initialization, learning expounds parallel independent modes and that certain regions of parameter space might lead to failed training. We also demonstrate that input norm and features' frequency in the dataset lead to distinct convergence speeds which might shed some light on the generalization capabilities of deep neural networks. We provide a comparison between the dynamics of learning with cross-entropy and hinge losses, which could prove useful to understand recent progress in the training of generative adversarial networks. Finally, we identify a phenomenon that we baptize \textit{gradient starvation} where the most frequent features in a dataset prevent the learning of other less frequent but equally informative features.
2018-09-17
ArXiv (preprint)
arxiv.org
arxiv.org
CNN Prediction of Future Disease Activity for Multiple Sclerosis Patients from Baseline MRI and Lesion Labels
Nazanin Mohammadi Sepahvand
Tal Hassner
Douglas Arnold
Tal Arbel
2018-09-15
BrainLes@MICCAI (published)
doi.org
3D U-Net for Brain Tumour Segmentation
Raghav Mehta
Tal Arbel
2018-09-15
BrainLes@MICCAI (published)
doi.org
How to Exploit Weaknesses in Biomedical Challenge Design and Organization
Annika Reinke
Matthias Eisenmann
Sinan Onogur
Marko Stankovic
Patrick Scholz
Peter M. Full
Hrvoje Bogunovic
Bennett Landman
Oskar Maier
Bjoern Menze
Gregory C. Sharp
Korsuk Sirinukunwattana
Stefanie Speidel
F. V. D. Sommen
Guoyan Zheng
Henning Müller
Michal Kozubek
Tal Arbel
Andrew P. Bradley
Pierre Jannin … (see 2 more)
Annette Kopp-Schneider
Lena Maier-Hein
2018-09-12
Medical Image Computing and Computer Assisted Intervention – MICCAI 2018 (published)
doi.org
RS-Net: Regression-Segmentation 3D CNN for Synthesis of Full Resolution Missing Brain MRI in the Presence of Tumours
Raghav Mehta
Tal Arbel
2018-09-11
Simulation and Synthesis in Medical Imaging (published)
doi.org
arxiv.org
Social-Affiliation Networks: Patterns and the SOAR Model
Dhivya Eswaran
Reihaneh Rabbany
Artur Dubrawski
Christos Faloutsos
2018-09-09
ECML/PKDD (published)
doi.org
Ghost Units Yield Biologically Plausible Backprop in Deep Neural Networks
Thomas Mesnard
Gaetan Vignoud
João Sacramento
Walter Senn
Yoshua Bengio
2018-09-04
2018 Conference on Cognitive Computational Neuroscience (published)
doi.org
arxiv.org
Quaternion Convolutional Neural Networks for End-to-End Automatic Speech Recognition
Titouan Parcollet
Ying Zhang
Mohamed Morchid
Chiheb Trabelsi
Georges Linarès
Renato De Mori
Yoshua Bengio
Recently, the connectionist temporal classification (CTC) model coupled with recurrent (RNN) or convolutional neural networks (CNN), made it… (see more) easier to train speech recognition systems in an end-to-end fashion. However in real-valued models, time frame components such as mel-filter-bank energies and the cepstral coefficients obtained from them, together with their first and second order derivatives, are processed as individual elements, while a natural alternative is to process such components as composed entities. We propose to group such elements in the form of quaternions and to process these quaternions using the established quaternion algebra. Quaternion numbers and quaternion neural networks have shown their efficiency to process multidimensional inputs as entities, to encode internal dependencies, and to solve many tasks with less learning parameters than real-valued models. This paper proposes to integrate multiple feature views in quaternion-valued convolutional neural network (QCNN), to be used for sequence-to-sequence mapping with the CTC model. Promising results are reported using simple QCNNs in phoneme recognition experiments with the TIMIT corpus. More precisely, QCNNs obtain a lower phoneme error rate (PER) with less learning parameters than a competing model based on real-valued CNNs.
2018-09-01
Interspeech 2018 (published)
doi.org
arxiv.org