Publications

Doubly Robust Off-Policy Actor-Critic Algorithms for Reinforcement Learning
Riashat Islam
Raihan Seraj
Samin Yeasar Arnob
Doina Precup
We study the problem of off-policy critic evaluation in several variants of value-based off-policy actor-critic algorithms. Off-policy actor… (voir plus)-critic algorithms require an off-policy critic evaluation step, to estimate the value of the new policy after every policy gradient update. Despite enormous success of off-policy policy gradients on control tasks, existing general methods suffer from high variance and instability, partly because the policy improvement depends on gradient of the estimated value function. In this work, we present a new way of off-policy policy evaluation in actor-critic, based on the doubly robust estimators. We extend the doubly robust estimator from off-policy policy evaluation (OPE) to actor-critic algorithms that consist of a reward estimator performance model. We find that doubly robust estimation of the critic can significantly improve performance in continuous control tasks. Furthermore, in cases where the reward function is stochastic that can lead to high variance, doubly robust critic estimation can improve performance under corrupted, stochastic reward signals, indicating its usefulness for robust and safe reinforcement learning.
2019-12-09
arXiv (prépublication)
doi.org
arxiv.org
Interactive Psychometrics for Autism with the Human Dynamic Clamp: Interpersonal Synchrony from Sensory-motor to Socio-cognitive Domains
Florence Baillin
Aline Lefebvre
Amandine Pedoux
Yann Beauxis
Denis-Alexander Engemann
Anna Maruani
Frederique Amsellem
Thomas Bourgeron
Richard Delorme
Guillaume Dumas
2019-12-05
(publié)
doi.org
Applying Knowledge Transfer for Water Body Segmentation in Peru
Jessenia Gonzalez Villarreal
Debjani Bhowmick
Cesar Beltran Castañon
Kris Sankaran
Yoshua Bengio
2019-12-01
arXiv (prépublication)
doi.org
arxiv.org
Detecting GAN generated errors
Xiru Zhu
Fengdi Che
Tianzi Yang
Tzuyang Yu
David Meger
Gregory Dudek
Despite an impressive performance from the latest GAN for generating hyper-realistic images, GAN discriminators have difficulty evaluating t… (voir plus)he quality of an individual generated sample. This is because the task of evaluating the quality of a generated image differs from deciding if an image is real or fake. A generated image could be perfect except in a single area but still be detected as fake. Instead, we propose a novel approach for detecting where errors occur within a generated image. By collaging real images with generated images, we compute for each pixel, whether it belongs to the real distribution or generated distribution. Furthermore, we leverage attention to model long-range dependency; this allows detection of errors which are reasonable locally but not holistically. For evaluation, we show that our error detection can act as a quality metric for an individual image, unlike FID and IS. We leverage Improved Wasserstein, BigGAN, and StyleGAN to show a ranking based on our metric correlates impressively with FID scores. Our work opens the door for better understanding of GAN and the ability to select the best samples from a GAN model.
2019-12-01
ArXiv (prépublication)
arxiv.org
arxiv.org
Artificial Intelligence Based Cloud Distributor (AI-CD): Probing Low Cloud Distribution with Generative Adversarial Neural Networks
T. Yuan
H. Song
David Hall
Victor Schmidt
Kris Sankaran
Yoshua Bengio
2019-11-30
(publié)
www.semanticscholar.org
Automated curriculum generation for Policy Gradients from Demonstrations
Anirudh Srinivasan
Dzmitry Bahdanau
Maxime Chevalier-Boisvert
Yoshua Bengio
2019-11-30
arXiv (prépublication)
doi.org
arxiv.org
Forgetting at biologically realistic levels of neurogenesis in a large-scale hippocampal model
Lina M. Tran
Sheena A. Josselyn
Blake A. Richards
Paul W. Frankland
2019-11-30
Behavioural Brain Research (publié)
doi.org
Visualizing structure and transitions in high-dimensional biological data
Kevin R. Moon
David van Dijk
Zheng Wang
Scott Gigante
Daniel B. Burkhardt
William S. Chen
Kristina Yim
Antonia van den Elzen
Matthew Hirn
Ronald R. Coifman
Natalia Ivanova
Guy Wolf
Smita Krishnaswamy
2019-11-30
Nature Biotechnology (publié)
doi.org
Deconstructing and reconstructing word embedding algorithms
Edward Daniel Newell
Kian Kenyon-Dean
Jackie CK Cheung
Uncontextualized word embeddings are reliable feature representations of words used to obtain high quality results for various NLP applicati… (voir plus)ons. Given the historical success of word embeddings in NLP, we propose a retrospective on some of the most well-known word embedding algorithms. In this work, we deconstruct Word2vec, GloVe, and others, into a common form, unveiling some of the necessary and sufficient conditions required for making performant word embeddings. We find that each algorithm: (1) fits vector-covector dot products to approximate pointwise mutual information (PMI); and, (2) modulates the loss gradient to balance weak and strong signals. We demonstrate that these two algorithmic features are sufficient conditions to construct a novel word embedding algorithm, Hilbert-MLE. We find that its embeddings obtain equivalent or better performance against other algorithms across 17 intrinsic and extrinsic datasets.
2019-11-28
ArXiv (prépublication)
arxiv.org
arxiv.org
3DGV: Immersive Exploration of 3D Genome Structures using Virtual Reality
Éric Zhang
Chrisostomos Drogaris
Antoine Gédon
Aaron Sossin
Rajae Faraj
Haifen Chen
Yan Cyr
Jacek Majewski
Mathieu Blanchette
Jérôme Waldispühl
The analysis of 3D genomic data is expected to revolutionize our understanding of genome organization and regulatory mechanisms. Yet, the co… (voir plus)mplex spatial organization of this information can be difficult to interpret with 2D viewers. Virtual Reality (VR) technologies offer an opportunity to rethink our methods to visualize and navigate 3D objects. In this paper, we introduce the Virtual Reality 3D Genome Viewer ( 3DGV ), an open platform to experiment and develop VR solutions to explore 3D genome structures. http://3dgv.cs.mcgill.ca/
2019-11-25
bioRxiv (prépublication)
doi.org
Driver perceptions of advanced driver assistance systems and safety
Sophie Le Page
Jason Millar
Kelly Selina Bronson
Shalaleh Rismani
AJung Moon
Advanced driver assistance systems (ADAS) are often used in the automotive industry to highlight innovative improvements in vehicle safety. … (voir plus)However, today it is unclear whether certain automation (e.g., adaptive cruise control, lane keeping, parking assist) increases safety of our roads. In this paper, we investigate driver awareness, use, perceived safety, knowledge, training, and attitudes toward ADAS with different automation systems/features. Results of our online survey (n=1018) reveal that there is a significant difference in frequency of use and perceived safety for different ADAS features. Furthermore, we find that at least 70% of drivers activate an ADAS feature"most or all of the time"when driving, yet we find that at least 40% of drivers report feeling that ADAS often compromises their safety when activated. We also find that most respondents learn how to use ADAS in their vehicles by trying it out on the road by themselves, rather than through any formal driver education and training. These results may mirror how certain ADAS features are often activated by default resulting in high usage rates. These results also suggest a lack of driver training and education for safely interacting with, and operating, ADAS, such as turning off systems/features. These findings contribute to a critical discussion about the overall safety implications of current ADAS, especially as they enable higher-level automation features to creep into personal vehicles without a lockstep response in training, regulation, and policy.
2019-11-24
ArXiv (prépublication)
arxiv.org
arxiv.org
Drivers' Awareness, Knowledge, and Use of Autonomous Driving Assistance Systems (ADAS) and Vehicle Automation
Kelly Selina Bronson
Sophie Le Page
Katherine M. Robinson
AJung Moon
Shalaleh Rismani
Jason Millar
Advanced driver assistance systems (ADAS) technologies in vehicles (e.g. park assist, lane change assist, emergency braking, etc.), which ta… (voir plus)ke over parts of the driving task of human drivers, are advancing at a disruptive pace and hold the potential to deliver many benefits to society. However, public understanding of ADAS systems, and driver training and licensing for using them, are lagging behind the fast-paced technological development, which could raise safety issues or slow the deployment of ADAS, thus offsetting their potential benefits. There is, therefore, a need to investigate issues related to public perception of ADAS in order to develop appropriate policies and governance structures which support innovation, and result in the smooth deployment and acceptance of appropriate ADAS for society. In this work we perform a quantitative public survey to better understand how the public's awareness and knowledge of ADAS technologies in their vehicles correlate to their use or engagement of those technologies. We find that up to 67% of participants never or rarely use optional ADAS in their vehicles (e.g. adaptive cruise control), where women were less likely than men to use ADAS even though women reported more awareness of ADAS in their vehicles, better training, and more willingness to pay for ADAS. By performing this analysis we hope to raise awareness around the public perception of current state-of-the-art in ADAS technologies. We also hope to flag concerns that answers to these questions might raise for the regulatory agencies, and manufacturers, responsible for bringing these technologies to market.
2019-11-24
arXiv.org (prépublication)
dblp.uni-trier.de