Publications

Revisiting Fundamentals of Experience Replay

William Fedus

Prajit Ramachandran

Rishabh Agarwal

Mark Rowland

Will Dabney

Experience replay is central to off-policy algorithms in deep reinforcement learning (RL), but there remain significant gaps in our understa… (see more)nding. We therefore present a systematic and extensive analysis of experience replay in Q-learning methods, focusing on two fundamental properties: the replay capacity and the ratio of learning updates to experience collected (replay ratio). Our additive and ablative studies upend conventional wisdom around experience replay -- greater capacity is found to substantially increase the performance of certain algorithms, while leaving others unaffected. Counterintuitively we show that theoretically ungrounded, uncorrected n-step returns are uniquely beneficial while other techniques confer limited benefit for sifting through larger memory. Separately, by directly controlling the replay ratio we contextualize previous observations in the literature and empirically measure its importance across a variety of deep RL algorithms. Finally, we conclude by testing a set of hypotheses on the nature of these performance benefits.

2020-11-21

Proceedings of the 37th International Conference on Machine Learning (published)

proceedings.mlr.press

arxiv.org

What can I do here? A Theory of Affordances in Reinforcement Learning

Khimya Khetarpal

Zafarali Ahmed

Gheorghe Comanici

David Abel

Doina Precup

2020-11-21

Proceedings of the 37th International Conference on Machine Learning (published)

proceedings.mlr.press

arxiv.org

An Effective Anti-Aliasing Approach for Residual Networks

Cristina Vasconcelos

Hugo Larochelle

Vincent Dumoulin

Nicolas Le Roux

Ross Goroshin

Image pre-processing in the frequency domain has traditionally played a vital role in computer vision and was even part of the standard pipe… (see more)line in the early days of deep learning. However, with the advent of large datasets, many practitioners concluded that this was unnecessary due to the belief that these priors can be learned from the data itself. Frequency aliasing is a phenomenon that may occur when sub-sampling any signal, such as an image or feature map, causing distortion in the sub-sampled output. We show that we can mitigate this effect by placing non-trainable blur filters and using smooth activation functions at key locations, particularly where networks lack the capacity to learn them. These simple architectural changes lead to substantial improvements in out-of-distribution generalization on both image classification under natural corruptions on ImageNet-C [10] and few-shot learning on Meta-Dataset [17], without introducing additional trainable parameters and using the default hyper-parameters of open source codebases.

2020-11-20

ArXiv (preprint)

arxiv.org

Multiscale PHATE Exploration of SARS-CoV-2 Data Reveals Multimodal Signatures of Disease

Manik Kuchroo

Jessie Huang

Patrick Wong

Jean-Christophe Grenier

Dennis Shung

Alexander Tong

Carolina Lucas

Jon Klein

Daniel B. Burkhardt

Scott Gigante

Abhinav Godavarthi

Benjamin Israelow

Tianyang Mao

Ji Eun Oh

Julio Silva

Takehiro Takahashi

Camila D. Odio

Arnau Casanovas-Massana

John Fournier

Shelli Farhadian … (see 7 more)

Charles S. Dela Cruz

Albert I. Ko

F. Perry Wilson

Julie Hussin

Guy Wolf

Akiko Iwasaki

Smita Krishnaswamy

2020-11-17

bioRxiv (preprint)

doi.org

Learning Inter-Modal Correspondence and Phenotypes From Multi-Modal Electronic Health Records

Kejing Yin

William K. Cheung

Benjamin Fung

Jonathan Poon

Non-negative tensor factorization has been shown a practical solution to automatically discover phenotypes from the electronic health record… (see more)s (EHR) with minimal human supervision. Such methods generally require an input tensor describing the inter-modal interactions to be pre-established; however, the correspondence between different modalities (e.g., correspondence between medications and diagnoses) can often be missing in practice. Although heuristic methods can be applied to estimate them, they inevitably introduce errors, and leads to sub-optimal phenotype quality. This is particularly important for patients with complex health conditions (e.g., in critical care) as multiple diagnoses and medications are simultaneously present in the records. To alleviate this problem and discover phenotypes from EHR with unobserved inter-modal correspondence, we propose the collective hidden interaction tensor factorization (cHITF) to infer the correspondence between multiple modalities jointly with the phenotype discovery. We assume that the observed matrix for each modality is marginalization of the unobserved inter-modal correspondence, which are reconstructed by maximizing the likelihood of the observed matrices. Extensive experiments conducted on the real-world MIMIC-III dataset demonstrate that cHITF effectively infers clinically meaningful inter-modal correspondence, discovers phenotypes that are more clinically relevant and diverse, and achieves better predictive performance compared with a number of state-of-the-art computational phenotyping models.

2020-11-12

ArXiv (preprint)

doi.org

arxiv.org

Using Open Source Licensing to Regulate the Assembly of LAWS: A Preliminary Analysis

Cheng Lin

AJung Moon

Lethal autonomous weapons (LAWS) are an emerging technology capable of automatically targeting and exercising lethal force. Many scholars an… (see more)d advocates have petitioned to ban the technology internationally for a myriad of reasons. However, there are practical challenges to implementing a ban. One such challenge is posed by the “intangible” nature of the software that LAWS depends on, which is incompatible with implementation mechanisms such as export control. Given the dual-use nature of software, and the fact that software is developed by teams of individuals, a number of soft governance mechanisms have been proposed to regulate this technology. In this paper, we investigate the feasibility of one particular approach: leveraging open source licenses as a means to prohibit the use of certain software in LAWS. This approach is largely motivated by the fact that open source software underpins all of technology, especially AI. Through a review of the recent tech activism and open source activism, we evaluate whether open source licenses can feasibly limit the use of open source software to only non-LAWS applications. We distill the current challenges facing “ethics-driven” open source licensing efforts into three main obstacles: the need for clarity of licensing language, the lack of enforceability of licenses, and the lack of cohesiveness of the open source community. We propose that addressing these factors are also success criteria for future anti-LAWS open source initiatives. We find that open source licenses provide more theoretical than practical promise in regulating LAWS, and conclude that cohesion in the open source community is the key to their potential practical success in the future.

2020-11-12

2020 IEEE International Symposium on Technology and Society (ISTAS) (published)

doi.org

Global Surveillance of COVID-19 by mining news media using a multi-source dynamic embedded topic model

Yue Li

Pratheeksha Nair

Zhi Wen

Imane Chafi

Anya Okhmatovskaia

Guido Powell

Yannan Shen

David Buckeridge

2020-11-10

Proceedings of the 11th ACM International Conference on Bioinformatics, Computational Biology and Health Informatics (published)

doi.org

On Posterior Collapse and Encoder Feature Dispersion in Sequence VAEs.

Teng Long

Yanshuai Cao

Jackie Cheung

Variational autoencoders (VAEs) hold great potential for modelling text, as they could in theory separate high-level semantic and syntactic … (see more)properties from local regularities of natural language. Practically, however, VAEs with autoregressive decoders often suffer from posterior collapse, a phenomenon where the model learns to ignore the latent variables, causing the sequence VAE to degenerate into a language model. In this paper, we argue that posterior collapse is in part caused by the lack of dispersion in encoder features. We provide empirical evidence to verify this hypothesis, and propose a straightforward fix using pooling. This simple technique effectively prevents posterior collapse, allowing model to achieve significantly better data log-likelihood than standard sequence VAEs. Comparing to existing work, our proposed method is able to achieve comparable or superior performances while being more computationally efficient.

2020-11-10

(published)

www.semanticscholar.org

Approximate Planning and Learning for Partially Observed Systems

Aditya Mahajan

2020-11-09

International Conference of Control, Dynamic Systems, and Robotics (published)

doi.org

Effectiveness of quarantine and testing to prevent COVID-19 transmission from arriving travelers

Russell Wa

David Buckeridge

2020-11-04