Publications

Editorial: Social Interaction in Neuropsychiatry
Victoria Leong
Danilo Bzdok
Frieder M. Paulus
Kevin Pelphrey
Elizabeth Redcay
Leonhard Schilbach
2021-04-28
Frontiers in Psychiatry (published)
doi.org
What is Going on Inside Recurrent Meta Reinforcement Learning Agents?
Safa Alver
Doina Precup
Recurrent meta reinforcement learning (meta-RL) agents are agents that employ a recurrent neural network (RNN) for the purpose of"learning a… (see more) learning algorithm". After being trained on a pre-specified task distribution, the learned weights of the agent's RNN are said to implement an efficient learning algorithm through their activity dynamics, which allows the agent to quickly solve new tasks sampled from the same distribution. However, due to the black-box nature of these agents, the way in which they work is not yet fully understood. In this study, we shed light on the internal working mechanisms of these agents by reformulating the meta-RL problem using the Partially Observable Markov Decision Process (POMDP) framework. We hypothesize that the learned activity dynamics is acting as belief states for such agents. Several illustrative experiments suggest that this hypothesis is true, and that recurrent meta-RL agents can be viewed as agents that learn to act optimally in partially observable environments consisting of multiple related tasks. This view helps in understanding their failure cases and some interesting model-based results reported in the literature.
2021-04-28
ArXiv (preprint)
arxiv.org
arxiv.org
Recovering the Wedge Modes Lost to 21-cm Foregrounds
Samuel Gagnon-Hartman
Yue Cui
Adrian Liu
Siamak Ravanbakhsh
One of the critical challenges facing imaging studies of the 21-cm signal at the Epoch of Reionization (EoR) is the separation of astrophysi… (see more)cal foreground contamination. These foregrounds are known to lie in a wedge-shaped region of
2021-04-21
Monthly Notices of the Royal Astronomical Society (unknown)
doi.org
arxiv.org
Gradient Masked Federated Optimization
Irene Tenison
Sreya Francis
Irina Rish
2021-04-20
ArXiv (preprint)
arxiv.org
arxiv.org
hBERT + BiasCorp - Fighting Racism on the Web
Olawale Moses Onabola
Zhuang Ma
Xie Yang
Benjamin Akera
Ibraheem Abdulrahman
Jia Xue
Dianbo Liu
Yoshua Bengio
Subtle and overt racism is still present both in physical and online communities today and has impacted many lives in different segments of … (see more)the society. In this short piece of work, we present how we’re tackling this societal issue with Natural Language Processing. We are releasing BiasCorp, a dataset containing 139,090 comments and news segment from three specific sources - Fox News, BreitbartNews and YouTube. The first batch (45,000 manually annotated) is ready for publication. We are currently in the final phase of manually labeling the remaining dataset using Amazon Mechanical Turk. BERT has been used widely in several downstream tasks. In this work, we present hBERT, where we modify certain layers of the pretrained BERT model with the new Hopfield Layer. hBert generalizes well across different distributions with the added advantage of a reduced model complexity. We are also releasing a JavaScript library 3 and a Chrome Extension Application, to help developers make use of our trained model in web applications (say chat application) and for users to identify and report racially biased contents on the web respectively
2021-04-18
OpenReview.net/Archive (published)
openreview.net
openreview.net
INFOSHIELD: Generalizable Information-Theoretic Human-Trafficking Detection
Meng-Chieh Lee
Catalina Vajiac
Aayushi Kulshrestha
Sacha Lévy
Namyong Park
Cara Jones
Reihaneh Rabbany
Christos Faloutsos
Given a million escort advertisements, how can we spot near-duplicates? Such micro-clusters of ads are usually signals of human trafficking.… (see more) How can we summarize them, visually, to convince law enforcement to act? Can we build a general tool that works for different languages? Spotting micro-clusters of near-duplicate documents is useful in multiple, additional settings, including spam-bot detection in Twitter ads, plagiarism, and more.We present INFOSHIELD, which makes the following contributions: (a) Practical, being scalable and effective on real data, (b) Parameter-free and Principled, requiring no user-defined parameters, (c) Interpretable, finding a document to be the cluster representative, highlighting all the common phrases, and automatically detecting "slots", i.e. phrases that differ in every document; and (d) Generalizable, beating or matching domain-specific methods in Twitter bot detection and human trafficking detection respectively, as well as being language-independent finding clusters in Spanish, Italian, and Japanese. Interpretability is particularly important for the anti human-trafficking domain, where law enforcement must visually inspect ads.Our experiments on real data show that INFOSHIELD correctly identifies Twitter bots with an F1 score over 90% and detects human-trafficking ads with 84% precision. Moreover, it is scalable, requiring about 8 hours for 4 million documents on a stock laptop.
2021-04-18
2021 IEEE 37th International Conference on Data Engineering (ICDE) (published)
doi.org
The Surprising Performance of Simple Baselines for Misinformation Detection
Kellin Pelrine
Jacob Danovitch
Reihaneh Rabbany
As social media becomes increasingly prominent in our day to day lives, it is increasingly important to detect informative content and preve… (see more)nt the spread of disinformation and unverified rumours. While many sophisticated and successful models have been proposed in the literature, they are often compared with older NLP baselines such as SVMs, CNNs, and LSTMs. In this paper, we examine the performance of a broad set of modern transformer-based language models and show that with basic fine-tuning, these models are competitive with and can even significantly outperform recently proposed state-of-the-art methods. We present our framework as a baseline for creating and evaluating new methods for misinformation detection. We further study a comprehensive set of benchmark datasets, and discuss potential data leakage and the need for careful design of the experiments and understanding of datasets to account for confounding variables. As an extreme case example, we show that classifying only based on the first three digits of tweet ids, which contain information on the date, gives state-of-the-art performance on a commonly used benchmark dataset for fake news detection --Twitter16. We provide a simple tool to detect this problem and suggest steps to mitigate it in future datasets.
2021-04-18
Proceedings of the Web Conference 2021 (published)
doi.org
arxiv.org
Ethics of Corporeal, Co-present Robots as Agents of Influence: a Review
AJung Moon
Shalaleh Rismani
H. V. D. Van der Loos
2021-04-13
Current Robotics Reports (published)
doi.org
Towards Causal Federated Learning For Enhanced Robustness and Privacy
Sreya Francis
Irene Tenison
Irina Rish
Federated Learning is an emerging privacy-preserving distributed machine learning approach to building a shared model by performing distribu… (see more)ted training locally on participating devices (clients) and aggregating the local models into a global one. As this approach prevents data collection and aggregation, it helps in reducing associated privacy risks to a great extent. However, the data samples across all participating clients are usually not independent and identically distributed (non-iid), and Out of Distribution(OOD) generalization for the learned models can be poor. Besides this challenge, federated learning also remains vulnerable to various attacks on security wherein a few malicious participating entities work towards inserting backdoors, degrading the generated aggregated model as well as inferring the data owned by participating entities. In this paper, we propose an approach for learning invariant (causal) features common to all participating clients in a federated learning setup and analyze empirically how it enhances the Out of Distribution (OOD) accuracy as well as the privacy of the final learned model.
2021-04-13
ArXiv (preprint)
arxiv.org
arxiv.org
An Analysis of the Adaptation Speed of Causal Models
Rémi Le Priol
Reza Babanezhad Harikandeh
Yoshua Bengio
Simon Lacoste-Julien
Consider a collection of datasets generated by unknown interventions on an unknown structural causal model …
2021-04-12
International Conference on Artificial Intelligence and Statistics (Poster)
doi.org
proceedings.mlr.press
Neural Function Modules with Sparse Arguments: A Dynamic Approach to Integrating Information across Layers
Alex Lamb
Anirudh Goyal
Agnieszka Słowik
Michael Mozer
Philippe Beaudoin
Yoshua Bengio
Feed-forward neural networks consist of a sequence of layers, in which each layer performs some processing on the information from the previ… (see more)ous layer. A downside to this approach is that each layer (or module, as multiple modules can operate in parallel) is tasked with processing the entire hidden state, rather than a particular part of the state which is most relevant for that module. Methods which only operate on a small number of input variables are an essential part of most programming languages, and they allow for improved modularity and code re-usability. Our proposed method, Neural Function Modules (NFM), aims to introduce the same structural capability into deep learning. Most of the work in the context of feed-forward networks combining top-down and bottom-up feedback is limited to classification problems. The key contribution of our work is to combine attention, sparsity, top-down and bottom-up feedback, in a flexible algorithm which, as we show, improves the results in standard classification, out-of-domain generalization, generative modeling, and learning representations in the context of reinforcement learning.
2021-04-12
International Conference on Artificial Intelligence and Statistics (Poster)
doi.org
proceedings.mlr.press
Science-Software Linkage: The Challenges of Traceability between Scientific Knowledge and Software Artifacts
Hideaki Hata
Jin L.C. Guo
Raula Gaikovina Kula
Christoph Treude
Although computer science papers are often accompanied by software artifacts, connecting research papers to their software artifacts and vic… (see more)e versa is not always trivial. First of all, there is a lack of well-accepted standards for how such links should be provided. Furthermore, the provided links, if any, often become outdated: they are affected by link rot when pre-prints are removed, when repositories are migrated, or when papers and repositories evolve independently. In this paper, we summarize the state of the practice of linking research papers and associated source code, highlighting the recent efforts towards creating and maintaining such links. We also report on the results of several empirical studies focusing on the relationship between scientific papers and associated software artifacts, and we outline challenges related to traceability and opportunities for overcoming these challenges.
2021-04-12
ArXiv (preprint)
arxiv.org
arxiv.org