Publications

Actor Critic with Differentially Private Critic
Jonathan Lebensold
William Hamilton
Borja Balle
Doina Precup
Reinforcement learning algorithms are known to be sample inefficient, and often performance on one task can be substantially improved by lev… (see more)eraging information (e.g., via pre-training) on other related tasks. In this work, we propose a technique to achieve such knowledge transfer in cases where agent trajectories contain sensitive or private information, such as in the healthcare domain. Our approach leverages a differentially private policy evaluation algorithm to initialize an actor-critic model and improve the effectiveness of learning in downstream tasks. We empirically show this technique increases sample efficiency in resource-constrained control problems while preserving the privacy of trajectories collected in an upstream task.
2019-10-13
ArXiv (preprint)
doi.org
arxiv.org
Deep learning for Aerosol Forecasting
Caleb Hoyne
S. Karthik Mukkavilli
David Meger
Reanalysis datasets combining numerical physics models and limited observations to generate a synthesised estimate of variables in an Earth … (see more)system, are prone to biases against ground truth. Biases identified with the NASA Modern-Era Retrospective Analysis for Research and Applications, Version 2 (MERRA-2) aerosol optical depth (AOD) dataset, against the Aerosol Robotic Network (AERONET) ground measurements in previous studies, motivated the development of a deep learning based AOD prediction model globally. This study combines a convolutional neural network (CNN) with MERRA-2, tested against all AERONET sites. The new hybrid CNN-based model provides better estimates validated versus AERONET ground truth, than only using MERRA-2 reanalysis.
2019-10-13
ArXiv (preprint)
arxiv.org
arxiv.org
Nash Games Among Stackelberg Leaders
Margarida Carvalho
Gabriele Dragotto
Felipe Feijoo
Andrea Lodi
Sriram Sankaranarayanan
We analyze Nash games played among leaders of Stackelberg games (NASP). We show it is Σ p 2 - hard to decide if the game has a mixed-strate… (see more)gy Nash equilibrium (MNE), even when there are only two leaders and each leader has one follower. We provide a finite time algorithm with a running time bounded by O (2 2 n ) which computes MNEs for NASP when it exists and returns infeasibility if no MNE exists. We also provide two ways to improve the algorithm which involves constructing a series of inner approximations (alternatively, outer approximations) to the leaders’ feasible region that will provably obtain the required MNE. Finally, we test our algorithms on a range of NASPs arising out of a game in the energy market, where countries act as Stackelberg leaders who play a Nash game, and the domestic producers act as the followers.
2019-10-13
arXiv.org (preprint)
dblp.uni-trier.de
Improving Pathological Structure Segmentation via Transfer Learning Across Diseases
Barleen Kaur
Paul Lemaitre
Raghav Mehta
Nazanin Mohammadi Sepahvand
Doina Precup
Douglas Arnold
Tal Arbel
2019-10-12
Domain Adaptation and Representation Transfer and Medical Image Learning with Less Labels and Imperfect Data (published)
doi.org
Early Prediction of Alzheimer's Disease Progression Using Variational Autoencoders
Sumana Basu
Konrad Wagstyl
Azar Zandifar
D. Collins
Adriana Romero
Doina Precup
2019-10-09
Lecture Notes in Computer Science (published)
doi.org
InfoMask: Masked Variational Latent Representation to Localize Chest Disease
Saeid Asgari Taghanaki
Mohammad Havaei
Tess Berthier
Francis Dutil
Lisa Di Jorio
Ghassan Hamarneh
Yoshua Bengio
2019-10-09
Lecture Notes in Computer Science (published)
doi.org
arxiv.org
Machine Learning Advantages in Canadian Astrophysics
Kim Venn
Sébastien Fabbro
Adrian Liu
Yashar Hezaveh
Laurence Perreault-Levasseur
Gwendolyn Eadie
Sara Ellison
Joanna Woo
JJ Kavelaars
Kwang Moo Yi
Renée Hložek
Jo Bovy
Hossen Teimoorinia
Siamak Ravanbakhsh
Locke Spencer
The application of machine learning (ML) methods to the analysis of astrophysical datasets is on the rise, particularly as the computing pow… (see more)er and complex algorithms become more powerful and accessible. As the field of ML enjoys a continuous stream of breakthroughs, its applications demonstrate the great potential of ML, ranging from achieving tens of millions of times increase in analysis speed (e.g., modeling of gravitational lenses or analysing spectroscopic surveys) to solutions of previously unsolved problems (e.g., foreground subtraction or efficient telescope operations). The number of astronomical publications that include ML has been steadily increasing since 2010.
With the advent of extremely large datasets from a new generation of surveys in the 2020s, ML methods will become an indispensable tool in astrophysics. Canada is an unambiguous world leader in the development of the field of machine learning, attracting large investments and skilled researchers to its prestigious AI Research Institutions. This provides a unique opportunity for Canada to also be a world leader in the application of machine learning in the field of astrophysics, and foster the training of a new generation of highly skilled researchers.
2019-10-01
ArXiv (preprint)
doi.org
arxiv.org
Reinforcement Learning Models of Human Behavior: Reward Processing in Mental Disorders
Baihan Lin
Guillermo Cecchi
Djallel Bouneffouf
Jenna Reinen
Irina Rish
Drawing an inspiration from behavioral studies of human decision making, we propose here a general parametric framework for a reinforcement … (see more)learning problem, which extends the standard Q-learning approach to incorporate a two-stream framework of reward processing with biases biologically associated with several neurological and psychiatric conditions, including Parkinson's and Alzheimer's diseases, attention-deficit/hyperactivity disorder (ADHD), addiction, and chronic pain. For the AI community, the development of agents that react differently to different types of rewards can enable us to understand a wide spectrum of multi-agent interactions in complex real-world socioeconomic systems. Empirically, the proposed model outperforms Q-Learning and Double Q-Learning in artificial scenarios with certain reward distributions and real-world human decision making gambling tasks. Moreover, from the behavioral modeling perspective, our parametric framework can be viewed as a first step towards a unifying computational model capturing reward processing abnormalities across multiple mental conditions and user preferences in long-term recommendation systems.
2019-10-01
NeurIPS.cc/2019/Workshop/Neuro_AI (poster)
openreview.net
openreview.net
Augmenting learning using symmetry in a biologically-inspired domain
Shruti Mishra
Abbas Abdolmaleki
Arthur Guez
Piotr Trochim
Doina Precup
Invariances to translation, rotation and other spatial transformations are a hallmark of the laws of motion, and have widespread use in the … (see more)natural sciences to reduce the dimensionality of systems of equations. In supervised learning, such as in image classification tasks, rotation, translation and scale invariances are used to augment training datasets. In this work, we use data augmentation in a similar way, exploiting symmetry in the quadruped domain of the DeepMind control suite (Tassa et al. 2018) to add to the trajectories experienced by the actor in the actor-critic algorithm of Abdolmaleki et al. (2018). In a data-limited regime, the agent using a set of experiences augmented through symmetry is able to learn faster. Our approach can be used to inject knowledge of invariances in the domain and task to augment learning in robots, and more generally, to speed up learning in realistic robotics applications.
2019-09-30
ArXiv (preprint)
arxiv.org
arxiv.org
Depth with Nonlinearity Creates No Bad Local Minima in ResNets
Kenji Kawaguchi
Yoshua Bengio
2019-09-30
Neural Networks (published)
doi.org
arxiv.org
Evaluation of a web-based tool for labelling potential hospital outbreaks: a mixed methods study
B. Leclère
David L Buckeridge
D. Lepelletier
2019-09-30
Journal of Hospital Infection (published)
doi.org
Learning Fixed Points in Generative Adversarial Networks: From Image-to-Image Translation to Disease Detection and Localization
Md Mahfuzur Rahman Siddiquee
Zongwei Zhou
Nima Tajbakhsh
Ruibin Feng
Michael B. Gotway
Yoshua Bengio
Jianming Liang
Generative adversarial networks (GANs) have ushered in a revolution in image-to-image translation. The development and proliferation of GANs… (see more) raises an interesting question: can we train a GAN to remove an object, if present, from an image while otherwise preserving the image? Specifically, can a GAN "virtually heal" anyone by turning his medical image, with an unknown health status (diseased or healthy), into a healthy one, so that diseased regions could be revealed by subtracting those two images? Such a task requires a GAN to identify a minimal subset of target pixels for domain translation, an ability that we call fixed-point translation, which no GAN is equipped with yet. Therefore, we propose a new GAN, called Fixed-Point GAN, trained by (1) supervising same-domain translation through a conditional identity loss, and (2) regularizing cross-domain translation through revised adversarial, domain classification, and cycle consistency loss. Based on fixed-point translation, we further derive a novel framework for disease detection and localization using only image-level annotation. Qualitative and quantitative evaluations demonstrate that the proposed method outperforms the state of the art in multi-domain image-to-image translation and that it surpasses predominant weakly-supervised localization methods in both disease detection and localization. Implementation is available at https://github.com/jlianglab/Fixed-Point-GAN.
2019-09-30
2019 IEEE/CVF International Conference on Computer Vision (ICCV) (published)
doi.org
arxiv.org