Publications

Grow-and-Clip: Informative-yet-Concise Evidence Distillation for Answer Explanation

Yuyan Chen

Yanghua Xiao

Bang Liu

Interpreting the predictions of existing Question Answering (QA) models is critical to many real-world intelligent applications, such as QA … (voir plus)systems for healthcare, education, and finance. However, existing QA models lack interpretability and provide no feedback or explanation for end-users to help them understand why a specific prediction is the answer to a question. In this research, we argue that the evidences of an answer is critical to enhancing the interpretability of QA models. Unlike previous research that simply extracts several sentence(s) in the context as evidence, we are the first to explicitly define the concept of evidence as the supporting facts in a context which are informative, concise, and readable. Besides, we provide effective strategies to quantitatively measure the informativeness, conciseness and readability of evidence. Furthermore, we propose Grow-and-Clip Evidence Distillation (GCED) algorithm to extract evidences from the contexts by trade-off informativeness, conciseness, and readability. We conduct extensive experiments on the SQuAD and TriviaQA datasets with several baseline models to evaluate the effect of GCED on interpreting answers to questions. Human evaluation are also carried out to check the quality of distilled evidences. Experimental results show that automatic distilled evidences have human-like informativeness, conciseness and readability, which can enhance the interpretability of the answers to questions.

2022-01-13

ArXiv (preprint)

Robust Contrastive Learning against Noisy Views

Ching-Yao Chuang

(Rex) Devon Hjelm

Xin Wang

Vibhav Vineet

Neel Joshi

Antonio Torralba

Stefanie Jegelka

Ya-heng Song

Contrastive learning relies on an assumption that positive pairs contain related views that share certain underlying information about an in… (voir plus)stance, e.g., patches of an image or co-occurring multimodal signals of a video. What if this assumption is violated? The literature suggests that contrastive learning produces suboptimal representations in the presence of noisy views, e.g., false positive pairs with no apparent shared information. In this work, we pro-pose a new contrastive loss function that is robust against noisy views. We provide rigorous theoretical justifications by showing connections to robust symmetric losses for noisy binary classification and by establishing a new contrastive bound for mutual information maximization based on the Wasserstein distance measure. The proposed loss is completely modality-agnostic and a simple drop-in replacement for the InfoNCE loss, which makes it easy to apply to ex-isting contrastive frameworks. We show that our approach provides consistent improvements over the state-of-the-art on image, video, and graph contrastive learning bench-marks that exhibit a variety of real-world noise patterns.

2022-01-12

ArXiv (preprint)

Magnetoencephalography resting-state correlates of executive and language components of verbal fluency

Victor Oswald

Younes Zerouali

Aubrée Boulet-Craig

Maja Krajinovic

Caroline Laverdière

Daniel Sinnett

Pierre Jolicoeur

Sarah Lippé

Karim Jerbi

Philippe Robaey

2022-01-10

Scientific Reports (publié)

Automated, interactive, and traceable domain modelling empowered by artificial intelligence

Rijul Saini

Gunter Mussbacher

Jin Guo

Jörg Kienzle

2022-01-08

Software and Systems Modeling (publié)

FIXME: synchronize with database! An empirical study of data access self-admitted technical debt

Biruk Asmare Muse

Csaba Zoltán Nagy

Anthony Cleve

Foutse Khomh

Giuliano Antoniol

2022-01-06

ArXiv (preprint)

A Generalized Bootstrap Target for Value-Learning, Efficiently Combining Value and Feature Predictions

Anthony GX-Chen

Veronica Chelu

Blake Richards

Joelle Pineau

Estimating value functions is a core component of reinforcement learning algorithms. Temporal difference (TD) learning algorithms use bootst… (voir plus)rapping, i.e. they update the value function toward a learning target using value estimates at subsequent time-steps. Alternatively, the value function can be updated toward a learning target constructed by separately predicting successor features (SF)—a policy-dependent model—and linearly combining them with instantaneous rewards. We focus on bootstrapping targets used when estimating value functions, and propose a new backup target, the ?-return mixture, which implicitly combines value-predictive knowledge (used by TD methods) with (successor) feature-predictive knowledge—with a parameter ? capturing how much to rely on each. We illustrate that incorporating predictive knowledge through an ??-discounted SF model makes more efficient use of sampled experience, compared to either extreme, i.e. bootstrapping entirely on the value function estimate, or bootstrapping on the product of separately estimated successor features and instantaneous reward models. We empirically show this approach leads to faster policy evaluation and better control performance, for tabular and nonlinear function approximations, indicating scalability and generality.

2022-01-05

ArXiv (preprint)

Global variation in event-based surveillance for disease outbreak detection: A time series analysis (Preprint)

Iris Ganser

R. Thiébaut

BACKGROUND Robust and flexible infectious disease surveillance is crucial for public health. Event-based surveillance (EBS) was developed t… (voir plus)o allow timely detection of infectious disease outbreaks using mostly web-based data. Despite its widespread use, EBS has not been evaluated systematically on a global scale in terms of outbreak detection performance. OBJECTIVE To assess the variation in timing and frequency of EBS reports compared to true outbreaks and to identify the determinants of variability, using the example of seasonal influenza epidemics in 24 countries. METHODS We obtained influenza-related reports from two EBS systems, HealthMap and the WHO Epidemic Intelligence from Open Sources (EIOS), and weekly virologic influenza counts from FluNet as a gold standard. Epidemic influenza periods were detected based on report frequency using Bayesian change point analysis. Timely sensitivity, i.e., outbreak detection within the first two weeks before or after an outbreak onset, was calculated along with sensitivity, specificity, positive predictive value, and timeliness of detection. Linear regressions were performed to assess the influence of country-specific factors on EBS performance. RESULTS Overall, monitoring the frequency of EBS reports detected 73.5% of outbreaks, but only 9.2% within two weeks of onset; in the best case, monitoring the frequency of health-related reports identified 29% of outbreaks within two weeks of onset. We observed a large degree of variability in all evaluation metrics across countries. The number of EBS reports available within a country, the human development index, and the country’s geographical location partially explained this variability. CONCLUSIONS Monitoring the frequency of EBS reports allowed just under 10% of seasonal influenza outbreaks to be detected in a timely manner in a worldwide analysis, with a large variability in detection capabilities. This article documents the global variation of EBS performance and demonstrates that monitoring report frequency alone in EBS may be insufficient for timely detection of outbreaks. Moreover, factors such as human development index and geographical location of a country may influence the performance of EBS and should be considered in future evaluations.

2022-01-05

(published)

Global variation in event-based surveillance for disease outbreak detection: A time series analysis (Preprint)

Iris Ganser

Rodolphe Thiébaut

BACKGROUND Robust and flexible infectious disease surveillance is crucial for public health. Event-based surveillance (EBS) was developed t… (voir plus)o allow timely detection of infectious disease outbreaks using mostly web-based data. Despite its widespread use, EBS has not been evaluated systematically on a global scale in terms of outbreak detection performance. OBJECTIVE To assess the variation in timing and frequency of EBS reports compared to true outbreaks and to identify the determinants of variability, using the example of seasonal influenza epidemics in 24 countries. METHODS We obtained influenza-related reports from two EBS systems, HealthMap and the WHO Epidemic Intelligence from Open Sources (EIOS), and weekly virologic influenza counts from FluNet as a gold standard. Epidemic influenza periods were detected based on report frequency using Bayesian change point analysis. Timely sensitivity, i.e., outbreak detection within the first two weeks before or after an outbreak onset, was calculated along with sensitivity, specificity, positive predictive value, and timeliness of detection. Linear regressions were performed to assess the influence of country-specific factors on EBS performance. RESULTS Overall, monitoring the frequency of EBS reports detected 73.5% of outbreaks, but only 9.2% within two weeks of onset; in the best case, monitoring the frequency of health-related reports identified 29% of outbreaks within two weeks of onset. We observed a large degree of variability in all evaluation metrics across countries. The number of EBS reports available within a country, the human development index, and the country’s geographical location partially explained this variability. CONCLUSIONS Monitoring the frequency of EBS reports allowed just under 10% of seasonal influenza outbreaks to be detected in a timely manner in a worldwide analysis, with a large variability in detection capabilities. This article documents the global variation of EBS performance and demonstrates that monitoring report frequency alone in EBS may be insufficient for timely detection of outbreaks. Moreover, factors such as human development index and geographical location of a country may influence the performance of EBS and should be considered in future evaluations.

2022-01-05

(publié)

Global Variations in Event-Based Surveillance for Disease Outbreak Detection: Time Series Analysis

Iris Ganser

Rodolphe Thiébaut

2022-01-05

JMIR Public Health and Surveillance (publié)

Global Variations in Event-Based Surveillance for Disease Outbreak Detection: Time Series Analysis

Iris Ganser

R. Thiébaut

Background Robust and flexible infectious disease surveillance is crucial for public health. Event-based surveillance (EBS) was developed to… (voir plus) allow timely detection of infectious disease outbreaks by using mostly web-based data. Despite its widespread use, EBS has not been evaluated systematically on a global scale in terms of outbreak detection performance. Objective The aim of this study was to assess the variation in the timing and frequency of EBS reports compared to true outbreaks and to identify the determinants of variability by using the example of seasonal influenza epidemic in 24 countries. Methods We obtained influenza-related reports between January 2013 and December 2019 from 2 EBS systems, that is, HealthMap and the World Health Organization Epidemic Intelligence from Open Sources (EIOS), and weekly virological influenza counts for the same period from FluNet as the gold standard. Influenza epidemic periods were detected based on report frequency by using Bayesian change point analysis. Timely sensitivity, that is, outbreak detection within the first 2 weeks before or after an outbreak onset was calculated along with sensitivity, specificity, positive predictive value, and timeliness of detection. Linear regressions were performed to assess the influence of country-specific factors on EBS performance. Results Overall, while monitoring the frequency of EBS reports over 7 years in 24 countries, we detected 175 out of 238 outbreaks (73.5%) but only 22 out of 238 (9.2%) within 2 weeks before or after an outbreak onset; in the best case, while monitoring the frequency of health-related reports, we identified 2 out of 6 outbreaks (33%) within 2 weeks of onset. The positive predictive value varied between 9% and 100% for HealthMap and from 0 to 100% for EIOS, and timeliness of detection ranged from 13% to 94% for HealthMap and from 0% to 92% for EIOS, whereas system specificity was generally high (59%-100%). The number of EBS reports available within a country, the human development index, and the country’s geographical location partially explained the high variability in system performance across countries. Conclusions We documented the global variation of EBS performance and demonstrated that monitoring the report frequency alone in EBS may be insufficient for the timely detection of outbreaks. In particular, in low- and middle-income countries, low data quality and report frequency impair the sensitivity and timeliness of disease surveillance through EBS. Therefore, advances in the development and evaluation and EBS are needed, particularly in low-resource settings.

2022-01-05

JMIR Public Health and Surveillance (published)

The epidemiological impact of the Canadian COVID Alert app

Shuo Sun

Mairead Shaw

Erica E. M. Moodie

Derek Ruths

We analyzed the effectiveness of the Canadian COVID Alert app on reducing COVID-19 infections and deaths due to the COVID-19 virus. Two sepa… (voir plus)rate but complementary approaches were taken. First, we undertook a comparative study to assess how the adoption and usage of the COVID Alert app compared to those of similar apps deployed in other regions. Next, we used data from the COVID Alert server and a range of plausible parameter values to estimate the numbers of infections and deaths averted in Canada using a model that combines information on number of notifications, secondary attack rate, expected fraction of transmissions that could be prevented, quarantine effectiveness, and expected size of the full transmission chain in the absence of exposure notification. The comparative analysis revealed that the COVID Alert app had among the lowest adoption levels among apps that reported usage. Our model indicates that use of the COVID Alert app averted between 6284 and 10,894 infections across the six Canadian provinces where app usage was highest during the March–July 2021 period. This range is equivalent to 1.6–2.9% of the total recorded infections across Canada in that time. Using province-specific case fatality rates, 57–101 deaths were averted during the same period. The number of cases and deaths averted was greatest in Ontario, whereas the proportion of cases and deaths averted was greatest in Newfoundland and Labrador. App impact measures were reported so rarely and so inconsistently by other regions that the relative assessment of impact is inconclusive. While the nationwide rates are low, provinces with widespread adoption of the app showed high ratios of averted cases and deaths (upper bound was greater than 60% of averted cases). This finding suggests that the COVID Alert app, when adopted at sufficient levels, can be an effective public health tool for combatting a pandemic such as COVID-19.

2022-01-05

Canadian journal of public health (published)