Publications

A Two-Stream Continual Learning System With Variational Domain-Agnostic Feature Replay
Qicheng Lao
Xiang Jiang
Mohammad Havaei
Yoshua Bengio
Learning in nonstationary environments is one of the biggest challenges in machine learning. Nonstationarity can be caused by either task dr… (voir plus)ift, i.e., the drift in the conditional distribution of labels given the input data, or the domain drift, i.e., the drift in the marginal distribution of the input data. This article aims to tackle this challenge with a modularized two-stream continual learning (CL) system, where the model is required to learn new tasks from a support stream and adapted to new domains in the query stream while maintaining previously learned knowledge. To deal with both drifts within and across the two streams, we propose a variational domain-agnostic feature replay-based approach that decouples the system into three modules: an inference module that filters the input data from the two streams into domain-agnostic representations, a generative module that facilitates the high-level knowledge transfer, and a solver module that applies the filtered and transferable knowledge to solve the queries. We demonstrate the effectiveness of our proposed approach in addressing the two fundamental scenarios and complex scenarios in two-stream CL.
2021-12-31
IEEE Transactions on Neural Networks and Learning Systems (publié)
doi.org
Understanding by Understanding Not: Modeling Negation in Language Models
Arian Hosseini
Siva Reddy
Dzmitry Bahdanau
R Devon Hjelm
Alessandro Sordoni
Aaron Courville
Negation is a core construction in natural language. Despite being very successful on many tasks, state-of-the-art pre-trained language mode… (voir plus)ls often handle negation incorrectly. To improve language models in this regard, we propose to augment the language modeling objective with an unlikelihood objective that is based on negated generic sentences from a raw text corpus. By training BERT with the resulting combined objective we reduce the mean top~1 error rate to 4% on the negated LAMA dataset. We also see some improvements on the negated NLI benchmarks.
2021-12-31
Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (publié)
doi.org
arxiv.org
Usefulness of School Absenteeism Data for Predicting Infl uenza Outbreaks,
Joseph R. Egger
A. Hoen
John S. Brownstein
David L Buckeridge
Donald R. Olson
Kevin James Konty
and second-round PCR were 94°C for 3 min, followed by 40 cycles of 94°C for 30 s, 55°C for 30 s, and 72°C for 2 min. Expected amplifi ca… (voir plus)tion products were 458 bp (PCR-1) and 304 bp (PCR-2). Using dilutions of a synthetic template corresponding to the target sequence, we estimated the sensitivity of the amplifi cation assay to be 5 copies of target sequence by limiting-dilution assay. Negative (sterile water) and positive controls (synthetic template dilutions) were
2021-12-31
(publié)
www.semanticscholar.org
Weakly-supervised Temporal Path Representation Learning with Contrastive Curriculum Learning — Extended Version
Sean Bin Yang
Chenjuan Guo
Jilin Hu
Bin Yang
Jian Tang
Christian S. Jensen
In step with the digitalization of transportation, we are witnessing a growing range of path-based smart-city applications, e.g., travel-tim… (voir plus)e estimation and travel path ranking. A temporal path(TP) that includes temporal information, e.g., departure time, into the path is fundamental to enable such applications. In this setting, it is essential to learn generic temporal path representations(TPRs) that consider spatial and temporal correlations simultaneously and that can be used in different applications, i.e., downstream tasks. Existing methods fail to achieve the goal since (i) supervised methods require large amounts of task-specific labels when training and thus fail to generalize the obtained TPRs to other tasks; (ii) through unsupervised methods can learn generic representations, they disregard the temporal aspect, leading to sub-optimal results. To contend with the limitations of existing solutions, we propose a Weakly-Supervised Contrastive (WSC) learning model. We first propose a temporal path encoder that encodes both the spatial and temporal information of a temporal path into a TPR. To train the encoder, we introduce weak labels that are easy and inexpensive to obtain and are relevant to different tasks, e.g., temporal labels indicating peak vs. off-peak hours from departure times. Based on the weak labels, we construct meaningful positive and negative temporal path samples by considering both spatial and temporal information, which facilities training the encoder using contrastive learning by pulling closer to the positive samples' representations while pushing away the negative samples' representations. To better guide contrastive learning, we propose a learning strategy based on Curriculum Learning such that the learning performs from easy to hard training instances. Experiments studies verify the effectiveness of the proposed method.
2021-12-31
IEEE International Conference on Data Engineering (inconnu)
doi.org
arxiv.org
What does it mean to be an AI Ethicist: An ontology of existing roles
Shalaleh Rismani
AJung Moon
With the increasing adoption of Artificial Intelligence systems (AIS) in various application and the growing efforts to regulate such system… (voir plus)s, a new set of occupations has emerged in the industry. This new set of roles take different titles and hold varying responsibilities. However, the individuals in these roles are tasked with interpreting and operationalizing best practices for developing ethical and safe AI systems. We will broadly refer to this new set of occupations as AI ethicists and recognize that they often hold a specific role in the intersection of technology development, business needs, and societal implications. In this work, we examine what it means to be an AI ethicist in the industry and propose an ontology of existing roles under this broad title along with their required competencies. We create this ontology by examining the job postings for such roles over the past two years and conduct expert interviews with fourteen individuals who currently hold such a role in the industry. The proposed ontology will inform executives and leaders who are looking to build responsible AI teams and provide educators the necessary information for creating new learning objectives and curriculum.
2021-12-31
arXiv.org (prépublication)
doi.org
Work-in-Progress: Utilizing latency and accuracy predictors for efficient hardware-aware NAS
Negin Firouzian
Seyyed Hasan Mozafari
James J. Clark
Warren J. Gross
Brett H. Meyer
With the increased size and complexity of state-of-the-art language models such as BERT, deploying them on resource-constrained devices has … (voir plus)become challenging. Latency-aware Neural Architecture Search (NAS) is an effective solution for finding an efficient implementation of complex models that satisfy hardware limitations. However, collecting on-device accuracy and latency feedback would significantly slow down the search process, making NAS impractical. To address this, we propose a low-cost method that models both accuracy and latency of BERT-based models on the target device, NVIDIA Jetson TX2, and removes the hardware-related delays from the search loop. Using a Random Forest regressor, our predictors outperform the state-of-the-art and achieve up to 57x speedup while finding a set of near-optimal models.
2021-12-31
International Conference on Hardware/Software Codesign and System Synthesis (publié)
doi.org
Adapting to the COVID‐19 pandemic in cohort studies: Validation of online assessments of cognition and neuropsychiatric symptoms in an aging population
Firoza Z Lussier
Stijn Servaes
Min Su Kang
Gleb Bezgin
Mira Chamoun
Jenna Stevenson
Nesrine Rahmouni
Alyssa Stevenson
Tharick A. Pascoal
Suzanne King
Danilo Bzdok
Serge Gauthier
Pedro Rosa‐Neto
The occurrence of the COVID‐19 pandemic has had a significant impact on cohort studies, particularly those whose subjects are at higher ri… (voir plus)sk of developing complications from the virus. As such, assessment methods must be adapted to minimize COVID‐19 exposure risk. The TRIAD (Translational Biomarkers of Aging and Dementia) cohort assessed N=292 individuals during initial COVID‐19 lockdown measures by telephone interview to rate cognition, neuropsychiatric symptoms, and impact of the pandemic. To increase speed and efficiency of data collection, we aim to follow these individuals by means of online survey. Here, we present a validation of our online assessment tools by comparing data obtained through both methods (phone interview and online survey) in the same subjects.
2021-12-30
Alzheimer's & Dementia (publié)
doi.org
Importance of Empirical Sample Complexity Analysis for Offline Reinforcement Learning
Samin Yeasar Arnob
Riashat Islam
Doina Precup
2021-12-30
ArXiv (prépublication)
arxiv.org
arxiv.org
Single-Shot Pruning for Offline Reinforcement Learning
Samin Yeasar Arnob
Riyasat Ohib
Sergey Plis
Doina Precup
2021-12-30
arXiv (prépublication)
doi.org
arxiv.org
Tau‐PET is associated with knowledge of COVID‐19, COVID‐19‐related distress, and change in sleep quality during the pandemic
Firoza Z Lussier
Stijn Servaes
Min Su Kang
Gleb Bezgin
Mira Chamoun
Jenna Stevenson
Nesrine Rahmouni
Alyssa Stevenson
Tharick A. Pascoal
Suzanne King
Guillaume Elgbeili
Danilo Bzdok
Serge Gauthier
Pedro Rosa‐Neto
While the global COVID‐19 pandemic has hindered many human research operations, it has allowed for the investigation of novel scientific q… (voir plus)uestions. Particularly, the effects of the pandemic and its resulting social isolation on elderly individuals and their association with Alzheimer’s disease biomarkers remains a broad and open question. Here, we sought to investigate whether knowledge of COVID‐19, pandemic‐related distress, and changes in sleep quality were associated with in vivo tau deposition in an AD‐enriched cohort.
2021-12-30
Alzheimer's & Dementia (publié)
doi.org
Multi-Domain Balanced Sampling Improves Out-of-Generalization of Chest X-ray Pathology Prediction Models
Enoch Amoatey Tetteh
Joseph Viviano
Yoshua Bengio
David M. Krueger
Joseph Paul Cohen
Learning models that generalize under different distribution shifts in medical imaging has been a long-standing research challenge. There ha… (voir plus)ve been several proposals for efficient and robust visual representation learning among vision research practitioners, especially in the sensitive and critical biomedical domain. In this paper, we propose an idea for out-of-distribution generalization of chest X-ray pathologies that uses a simple balanced batch sampling technique. We observed that balanced sampling between the multiple training datasets improves the performance over baseline models trained without balancing. Code for this work is available on Github. 1
2021-12-26
arXiv (prépublication)
www.semanticscholar.org
Multi-Domain Balanced Sampling Improves Out-of-Distribution Generalization of Chest X-ray Pathology Prediction Models
Enoch Tetteh
Joseph Viviano
Yoshua Bengio
David Krueger
Joseph Paul Cohen
Learning models that generalize under different distribution shifts in medical imaging has been a long-standing research challenge. There ha… (voir plus)ve been several proposals for efficient and robust visual representation learning among vision research practitioners, especially in the sensitive and critical biomedical domain. In this paper, we propose an idea for out-of-distribution generalization of chest X-ray pathologies that uses a simple balanced batch sampling technique. We observed that balanced sampling between the multiple training datasets improves the performance over baseline models trained without balancing.
2021-12-26
arXiv (prépublication)
doi.org
arxiv.org