Publications

The KITMUS Test: Evaluating Knowledge Integration from Multiple Sources
Akshatha Arodi
Martin Pömsl
Kaheer Suleman
Adam Trischler
A.R. Olteanu
Jackie CK Cheung
Many state-of-the-art natural language understanding (NLU) models are based on pretrained neural language models. These models often make in… (see more)ferences using information from multiple sources. An important class of such inferences are those that require both background knowledge, presumably contained in a model’s pretrained parameters, and instance-specific information that is supplied at inference time. However, the integration and reasoning abilities of NLU models in the presence of multiple knowledge sources have been largely understudied. In this work, we propose a test suite of coreference resolution subtasks that require reasoning over multiple facts. These subtasks differ in terms of which knowledge sources contain the relevant facts. We also introduce subtasks where knowledge is present only at inference time using fictional knowledge. We evaluate state-of-the-art coreference resolution models on our dataset. Our results indicate that several models struggle to reason on-the-fly over knowledge observed both at pretrain time and at inference time. However, with task-specific training, a subset of models demonstrates the ability to integrate certain knowledge types from multiple sources. Still, even the best performing models seem to have difficulties with reliably integrating knowledge presented only at inference time.
2022-12-31
ACL (1) (published)
doi.org
arxiv.org
On the Limitations of Elo: Real-World Games, are Transitive, not Additive
Quentin Bertrand
Wojciech M. Czarnecki
Gauthier Gidel
Real-world competitive games, such as chess, go, or StarCraft II, rely on Elo models to measure the strength of their players. Since these g… (see more)ames are not fully transitive, using Elo implicitly assumes they have a strong transitive component that can correctly be identified and extracted. In this study, we investigate the challenge of identifying the strength of the transitive component in games. First, we show that Elo models can fail to extract this transitive component, even in elementary transitive games. Then, based on this observation, we propose an extension of the Elo score: we end up with a disc ranking system that assigns each player two scores, which we refer to as skill and consistency. Finally, we propose an empirical validation on payoff matrices coming from real-world games played by bots and humans.
2022-12-31
AISTATS (published)
doi.org
arxiv.org
The race to understand immunopathology in COVID-19: Perspectives on the impact of quantitative approaches to understand within-host interactions
Sonia Gazeau
Xiaoyan Deng
Hsu Kiang Ooi
Fatima Mostefai
Julie G Hussin
Jane Heffernan
Adrianne L. Jenner
Morgan Craig
2022-12-31
ImmunoInformatics (published)
doi.org
The StatCan Dialogue Dataset: Retrieving Data Tables through Conversations with Genuine Intents
Xing Han Lu
Siva Reddy
Harm de Vries
We introduce the StatCan Dialogue Dataset consisting of 19,379 conversation turns between agents working at Statistics Canada and online use… (see more)rs looking for published data tables. The conversations stem from genuine intents, are held in English or French, and lead to agents retrieving one of over 5000 complex data tables. Based on this dataset, we propose two tasks: (1) automatic retrieval of relevant tables based on a on-going conversation, and (2) automatic generation of appropriate agent responses at each turn. We investigate the difficulty of each task by establishing strong baselines. Our experiments on a temporal data split reveal that all models struggle to generalize to future conversations, as we observe a significant drop in performance across both tasks when we move from the validation to the test set. In addition, we find that response generation models struggle to decide when to return a table. Considering that the tasks pose significant challenges to existing models, we encourage the community to develop models for our task, which can be directly used to help knowledge workers find relevant tables for live chat users.
2022-12-31
EACL (published)
doi.org
arxiv.org
The Statistical Benefits of Quantile Temporal-Difference Learning for Value Estimation
Mark Rowland
Yunhao Tang
Clare Lyle
Remi Munos
Bellemare Marc-Emmanuel
Will Dabney
2022-12-31
ICML (published)
doi.org
proceedings.mlr.press
A Theory of Continuous Generative Flow Networks
Salem Lahlou
Tristan Deleu
Pablo Lemos
Dinghuai Zhang
Alexandra Volokhova
Alex Hernández-García
Lena Nehale Ezzine
Yoshua Bengio
Nikolay Malkin
Generative flow networks (GFlowNets) are amortized variational inference algorithms that are trained to sample from unnormalized target dist… (see more)ributions over compositional objects. A key limitation of GFlowNets until this time has been that they are restricted to discrete spaces. We present a theory for generalized GFlowNets, which encompasses both existing discrete GFlowNets and ones with continuous or hybrid state spaces, and perform experiments with two goals in mind. First, we illustrate critical points of the theory and the importance of various assumptions. Second, we empirically demonstrate how observations about discrete GFlowNets transfer to the continuous case and show strong results compared to non-GFlowNet baselines on several previously studied tasks. This work greatly widens the perspectives for the application of GFlowNets in probabilistic inference and various modeling settings.
2022-12-31
ICML (published)
doi.org
proceedings.mlr.press
Toward computing attributions for dimensionality reduction techniques
Matthew Scicluna
Jean-Christophe Grenier
Raphaël Poujol
Sébastien Lemieux
Julie G. Hussin
We describe the problem of computing local feature attributions for dimensionality reduction methods. We use one such method that is well es… (see more)tablished within the context of supervised classification—using the gradients of target outputs with respect to the inputs—on the popular dimensionality reduction technique t-SNE, widely used in analyses of biological data. We provide an efficient implementation for the gradient computation for this dimensionality reduction technique. We show that our explanations identify significant features using novel validation methodology; using synthetic datasets and the popular MNIST benchmark dataset. We then demonstrate the practical utility of our algorithm by showing that it can produce explanations that agree with domain knowledge on a SARS-CoV-2 sequence dataset. Throughout, we provide a road map so that similar explanation methods could be applied to other dimensionality reduction techniques to rigorously analyze biological datasets. We have created a Python package that can be installed using the following command: pip install interpretable_tsne. All code used can be found at github.com/MattScicluna/interpretable_tsne.
2022-12-31
Bioinformatics Advances (published)
doi.org
Towards Detecting Contextual Real-Time Toxicity for In-Game Chat
Zachary Yang
Nicolas Grenon-Godbout
Reihaneh Rabbany
Real-time toxicity detection in online environments poses a significant challenge, due to the increasing prevalence of social media and gami… (see more)ng platforms. We introduce ToxBuster, a simple and scalable model that reliably detects toxic content in real-time for a line of chat by including chat history and metadata. ToxBuster consistently outperforms conventional toxicity models across popular multiplayer games, including Rainbow Six Siege, For Honor, and DOTA 2. We conduct an ablation study to assess the importance of each model component and explore ToxBuster's transferability across the datasets. Furthermore, we showcase ToxBuster's efficacy in post-game moderation, successfully flagging 82.1% of chat-reported players at a precision level of 90.0%. Additionally, we show how an additional 6% of unreported toxic players can be proactively moderated.
2022-12-31
EMNLP (Findings) (published)
doi.org
openreview.net
Towards Learning to Imitate from a Single Video Demonstration
Glen Berseth
Florian Golemo
Christopher Pal
Agents that can learn to imitate given video observation -- \emph{without direct access to state or action information} are more applicable … (see more)to learning in the natural world. However, formulating a reinforcement learning (RL) agent that facilitates this goal remains a significant challenge. We approach this challenge using contrastive training to learn a reward function comparing an agent's behaviour with a single demonstration. We use a Siamese recurrent neural network architecture to learn rewards in space and time between motion clips while training an RL policy to minimize this distance. Through experimentation, we also find that the inclusion of multi-task data and additional image encoding losses improve the temporal consistency of the learned rewards and, as a result, significantly improves policy learning. We demonstrate our approach on simulated humanoid, dog, and raptor agents in 2D and a quadruped and a humanoid in 3D. We show that our method outperforms current state-of-the-art techniques in these environments and can learn to imitate from a single video demonstration.
2022-12-31
J. Mach. Learn. Res. (published)
doi.org
arxiv.org
Towards Reliable Neural Specifications
Chuqin Geng
Nham Le
Xiaojie Xu
Zhaoyue Wang
Arie Gurfinkel
Xujie Si
2022-12-31
ICML (published)
proceedings.mlr.press
proceedings.mlr.press
TrafficVis: Visualizing Organized Activity and Spatio-Temporal Patterns for Detecting and Labeling Human Trafficking
Catalina Vajiac
Duen Horng Chau
Andreas Olligschlaeger
Rebecca Mackenzie
Pratheeksha Nair
Meng-Chieh Lee
Yifei Li
Namyong Park
Reihaneh Rabbany
Christos Faloutsos
Law enforcement and domain experts can detect human trafficking (HT) in online escort websites by analyzing suspicious clusters of connected… (see more) ads. How can we explain clustering results intuitively and interactively, visualizing potential evidence for experts to analyze? We present TrafficVis, the first interface for cluster-level HT detection and labeling. Developed through months of participatory design with domain experts, TrafficVis provides coordinated views in conjunction with carefully chosen backend algorithms to effectively show spatio-temporal and text patterns to a wide variety of anti-HT stakeholders. We build upon state-of-the-art text clustering algorithms by incorporating shared metadata as a signal of connected and possibly suspicious activity, then visualize the results. Domain experts can use TrafficVis to label clusters as HT, or other, suspicious, but non-HT activity such as spam and scam, quickly creating labeled datasets to enable further HT research. Through domain expert feedback and a usage scenario, we demonstrate TRAFFICVIS's efficacy. The feedback was overwhelmingly positive, with repeated high praises for the usability and explainability of our tool, the latter being vital for indicting possible criminals.
2022-12-31
IEEE Transactions on Visualization and Computer Graphics (published)
doi.org
Transposable elements regulate thymus development and function 1
Jean-David Larouche
Céline M. Laumont
Assya Trofimov
Krystel Vincent
Leslie Hesnard
Sylvie Brochu
Caroline Côté
Juliette Humeau
Éric Bonneil
Joël Lanoix
Chantal Durette
Patrick Gendron
Jean-Philippe Laverdure
Ellen Rothman Richie
S. Lemieux
Pierre Thibault
Claude Perreault
21 Transposable elements (TE) are repetitive sequences representing ~45% of the human and mouse genomes 22 and are highly expressed by medul… (see more)lary thymic epithelial cells (mTEC). In this study, we investigated the 23 role of transposable elements (TE), which are highly expressed by medullary thymic epithelial cells 24 (mTEC), on T-cell development in the thymus. We performed multi-omic analyses of TEs in human and 25 mouse thymic cells to elucidate their role in T cell development. We report that TE expression in the 26 human thymus is high and shows extensive ageand cell lineage-related variations. TEs interact with 27 multiple transcription factors in all cell types of the human thymus. Two cell types express particularly 28 broad TE repertoires: mTECs and plasmacytoid dendritic cells (pDC). In mTECs, TEs interact with 29 transcription factors essential for mTEC development and function (e.g., PAX1 and RELB) and generate 30 MHC-I-associated peptides implicated in thymocyte education. Notably, AIRE, FEZF2, and CHD4 31 regulate non-redundant sets of TEs in murine mTECs. Human thymic pDCs homogenously express large 32 numbers of TEs that lead to the formation of dsRNA, triggering RIG-I and MDA5 signaling and 33 explaining why thymic pDCs constitutively secrete IFN ɑ/β. This study illustrates the diversity of 34 interactions between TEs and the adaptive immune system. TEs are genetic parasites, and the two thymic 35 cell types most affected by TEs (mTEcs and pDCs) are essential to establishing central T-cell tolerance. 36 Therefore, we propose that the orchestration of TE expression in thymic cells is critical to prevent 37 autoimmunity in vertebrates. 38
2022-12-31
(published)
www.semanticscholar.org