Representation of Reinforcement Learning Policies in Reproducing Kernel Hilbert Spaces.
Bogdan Mazoure
Thang Doan
Tianyu Li
Vladimir Makarenkov
Joelle Pineau
Doina Precup
Guillaume Rabusseau
We propose a general framework for policy representation for reinforcement learning tasks. This framework involves finding a low-dimensional… (see more) embedding of the policy on a reproducing kernel Hilbert space (RKHS). The usage of RKHS based methods allows us to derive strong theoretical guarantees on the expected return of the reconstructed policy. Such guarantees are typically lacking in black-box models, but are very desirable in tasks requiring stability. We conduct several experiments on classic RL domains. The results confirm that the policies can be robustly embedded in a low-dimensional space while the embedded policy incurs almost no decrease in return.
Cybersanté : les tentatives juridiques pour objectiver un domaine en pleine effervescence
Vincent Gautrais
Catherine Régis
Resting-state connectivity stratifies premanifest Huntington’s disease by longitudinal cognitive decline rate
Pablo Polosecki
Eduardo Castro
Irina Rish
Dorian Pustina
John H. Warner
Andrew Wood
Cristina Sampaio
Guillermo Cecchi
2020-01-27
Scientific Reports (published)
doi.org
Using Simulated Data to Generate Images of Climate Change
Gautier Cosne
Adrien Juraver
Mélisande Teng
Victor Schmidt
Vahe Vardanyan
Alexandra Luccioni
Yoshua Bengio
Generative adversarial networks (GANs) used in domain adaptation tasks have the ability to generate images that are both realistic and perso… (see more)nalized, transforming an input image while maintaining its identifiable characteristics. However, they often require a large quantity of training data to produce high-quality images in a robust way, which limits their usability in cases when access to data is limited. In our paper, we explore the potential of using images from a simulated 3D environment to improve a domain adaptation task carried out by the MUNIT architecture, aiming to use the resulting images to raise awareness of the potential future impacts of climate change.
2020-01-26
ArXiv (preprint)
arxiv.org
arxiv.org
Autism spectrum disorder
Catherine Lord
Traolach S. Brugha
Tony Charman
James Cusack
Guillaume Dumas
Thomas Frazier
Emily J. H. Jones
Rebecca M. Jones
Andrew Pickles
Matthew W. State
Julie Lounds Taylor
Jeremy Veenstra-VanderWeele
2020-01-16
Nature Reviews Disease Primers (published)
doi.org
Optogenetic activation of parvalbumin and somatostatin interneurons selectively restores theta-nested gamma oscillations and oscillation-induced spike timing-dependent long-term potentiation impaired by amyloid β oligomers
Kyerl Park
Jaedong Lee
Hyun Jae Jang
Blake Richards
Michael M Kohl
Jeehyun Kwag
2020-01-15
BMC Biology (published)
doi.org
Conducting gender-based analysis of existing databases when self-reported gender data are unavailable: the GENDER Index in a working population
M. Gabrielle Pagé
Bilkis Vissandjée
Hermine Lore Nguena Nguefack
Joel Katz
Oumar Mallé Samb
Alain Gillian Lucie David Manon Catherine Anaïs Benoit Alexandre Amélie Pasquale Valérie Marie-Pascale Mike Anne-Marie Marc Josiane Mireille Stéphanie Pierre Annie Isabelle Danielle Denis Jaime André Geneviève Jean-François Roxanne Marc-Antoine Pier Sonia Vanasse
Alain Gillian Lucie David Manon Catherine Anaïs Benoit A Vanasse Bartlett Blais Buckeridge Choinière Hudon
Alain Vanasse
Gillian Bartlett
Lucie Blais
David Buckeridge
Manon Choinière
Catherine Hudon
Anaïs Lacasse
Benoit Lamarche
Alexandre Lebel
Amélie Quesnel-Vallée
Pasquale Roberge
Valérie Émond
Marie-Pascale Pomey … (see 19 more)
Mike Benigeri
Anne-Marie Cloutier
Marc Dorais
Josiane Courteau
Mireille Courteau
Stéphanie Plante
Pierre Cambon
Annie Giguère
Isabelle Leroux
Danielle St-Laurent
Denis Roy
Jaime Borja
André Néron
Geneviève Landry
Jean-François Ethier
Roxanne Dault
Marc-Antoine Côté-Marcil
Pier Tremblay
Sonia Quirion
Objectives Growing attention has been given to considering sex and gender in health research. However, this remains a challenge in the conte… (see more)xt of retrospective studies where self-reported gender measures are often unavailable. This study aimed to create and validate a composite gender index using data from the Canadian Community Health Survey (CCHS). Methods According to scientific literature and expert opinion, the GENDER Index was built using several variables available in the CCHS and deemed to be gender-related (e.g., occupation, receiving child support, number of working hours). Among workers aged 18–50 years who had no missing data for our variables of interest ( n  = 29,470 participants), propensity scores were derived from a logistic regression model that included gender-related variables as covariates and where biological sex served as the dependent variable. Construct validity of propensity scores (GENDER Index scores) were then examined. Results When looking at the distribution of the GENDER Index scores in males and females, they appeared related but partly independent. Differences in the proportion of females appeared between groups categorized according to the GENDER Index scores tertiles ( p   0.0001). Construct validity was also examined through associations between the GENDER Index scores and gender-related variables identifi
2020-01-13
Canadian journal of public health (published)
doi.org
Accelerating Smooth Games by Manipulating Spectral Shapes
Waiss Azizian
Damien Scieur
Ioannis Mitliagkas
Simon Lacoste-Julien
Gauthier Gidel
We use matrix iteration theory to characterize acceleration in smooth games. We define the spectral shape of a family of games as the set co… (see more)ntaining all eigenvalues of the Jacobians of standard gradient dynamics in the family. Shapes restricted to the real line represent well-understood classes of problems, like minimization. Shapes spanning the complex plane capture the added numerical challenges in solving smooth games. In this framework, we describe gradient-based methods, such as extragradient, as transformations on the spectral shape. Using this perspective, we propose an optimal algorithm for bilinear games. For smooth and strongly monotone operators, we identify a continuum between convex minimization, where acceleration is possible using Polyak's momentum, and the worst case where gradient descent is optimal. Finally, going beyond first-order methods, we propose an accelerated version of consensus optimization.
2020-01-02
ArXiv (preprint)
arxiv.org
arxiv.org
On Bonus Based Exploration Methods In The Arcade Learning Environment
Adrien Ali Taiga
William Fedus
Marlos C. Machado
Aaron Courville
Marc Gendron-Bellemare
Research on exploration in reinforcement learning, as applied to Atari 2600 game-playing, has emphasized tackling difficult exploration prob… (see more)lems such as Montezuma's Revenge (Bellemare et al., 2016). Recently, bonus-based exploration methods, which explore by augmenting the environment reward, have reached above-human average performance on such domains. In this paper we reassess popular bonus-based exploration methods within a common evaluation framework. We combine Rainbow (Hessel et al., 2018) with different exploration bonuses and evaluate its performance on Montezuma's Revenge, Bellemare et al.'s set of hard of exploration games with sparse rewards, and the whole Atari 2600 suite. We find that while exploration bonuses lead to higher score on Montezuma's Revenge they do not provide meaningful gains over the simpler epsilon-greedy scheme. In fact, we find that methods that perform best on that game often underperform epsilon-greedy on easy exploration Atari 2600 games. We find that our conclusions remain valid even when hyperparameters are tuned for these easy-exploration games. Finally, we find that none of the methods surveyed benefit from additional training samples (1 billion frames, versus Rainbow's 200 million) on Bellemare et al.'s hard exploration games. Our results suggest that recent gains in Montezuma's Revenge may be better attributed to architecture change, rather than better exploration schemes; and that the real pace of progress in exploration research for Atari 2600 games may have been obfuscated by good results on a single domain.
2020-01-01
ICLR.cc/2020/Conference (poster)
openreview.net
openreview.net
Call for Papers: Novel Informatics Approaches to COVID-19 Research
Hua Xu
David Buckeridge
Yi Wang
Coping With Simulators That Don't Always Return
Andrew Warrington
Saeid Naderiparizi
Frank Wood
2020-01-01
AISTATS (published)
proceedings.mlr.press
arxiv.org
COVI White Paper-Version 1.1
Hannah Alsdurf
Yoshua Bengio
Tristan Deleu
Prateek Gupta
Daphne Ippolito
Richard Janda
Max Jarvie
Tyler J. Kolody
Sekoul Krastev
Tegan Maharaj
Robert Obryk
Dan Pilat
Valerie Pisano
Benjamin Prud'homme
Meng Qu
Nasim Rahaman
Irina Rish
Jean-franois Rousseau
Abhinav Sharma
Brooke Struck … (see 3 more)
Jian Tang
Martin Weiss
Yun William Yu
The SARS-CoV-2 (Covid-19) pandemic has resulted in significant strain on health care and public health institutions around the world. Contac… (see more)t tracing is an essential tool for public health officials and local communities to change the course of the Covid-19 pandemic. Standard manual contact tracing of people infected with Covid-19, while the current gold standard, has significant challenges that limit the ability of public health authorities to minimize community infections. Personalized peer-to-peer contact tracing through the use of mobile applications has the potential to shift the paradigm of Covid-19 community spread. Although some countries have deployed centralized tracking systems through either GPS or Bluetooth, more privacy-protecting decentralized systems offer much of the same benefit without concentrating data in the hands of a state authority or in for-profit corporations. Additionally, machine learning methods can be used to circumvent some of the limitations of standard digital tracing by incorporating many clues (including medical conditions, self-reported symptoms, and numerous encounters with people at different risk levels, for different durations and distances) and their uncertainty into a more graded and precise estimation of infection and contagion risk. The estimated risk can be used to provide early risk awareness, personalized recommendations and relevant information to the user and connect them to health services. Finally, the non-identifying data about these risks can inform detailed epidemiological models trained jointly with the machine learning predictor, and these models can provide statistical evidence for the interaction and importance of different factors involved in the transmission of the disease. They can also be used to monitor, evaluate and optimize different health policy and confinement/deconfinement scenarios according to medical and economic productivity indicators. However, such a strategy based on mobile apps and machine learning should proactively mitigate potential ethical and privacy risks, which could have substantial impacts on society (not only impacts on health but also impacts such as stigmatization and abuse of personal data). Here, we present an overview of the rationale, design, ethical considerations and privacy strategy of ‘COVI,’ a Covid-19 public peer-to-peer contact tracing and risk awareness mobile application developed in Canada. Addendum 2020-07-14: The government of Canada has declined to endorse COVI and will be promoting a different app for decentralized contact tracing. In the interest of preventing fragmentation of the app landscape, COVI will therefore not be deployed to end users. We are currently still in the process of finalizing the project, and plan to release our code and models for academic consumption and to make them accessible to other States should they wish to deploy an app based on or inspired by said code and models. University of Ottawa, Mila, Université de Montréal, The Alan Turing Institute, University of Oxford, University of Pennsylvania, McGill University, Borden Ladner Gervais LLP, The Decision Lab, HEC Montréal, Max Planck Institute, Libéo, University of Toronto. Corresponding author general: richard.janda@mcgill.ca Corresponding author for public health: abhinav.sharma@mcgill.ca Corresponding author for privacy: ywyu@math.toronto.edu Corresponding author for machine learning: yoshua.bengio@mila.quebec Corresponding author for user perspective: brooke@thedecisionlab.com Corresponding author for technical implementation: jean-francois.rousseau@libeo.com 1 ar X iv :2 00 5. 08 50 2v 2 [ cs .C R ] 2 7 Ju l 2 02 0