Publications

SAND-mask: An Enhanced Gradient Masking Strategy for the Discovery of Invariances in Domain Generalization

A major bottleneck in the real-world applications of machine learning models is their failure in generalizing to unseen domains whose data d… (see more)istribution is not i.i.d to the training domains. This failure often stems from learning non-generalizable features in the training domains that are spuriously correlated with the label of data. To address this shortcoming, there has been a growing surge of interest in learning good explanations that are hard to vary, which is studied under the notion of Out-of-Distribution (OOD) Generalization. The search for good explanations that are \textit{invariant} across different domains can be seen as finding local (global) minimas in the loss landscape that hold true across all of the training domains. In this paper, we propose a masking strategy, which determines a continuous weight based on the agreement of gradients that flow in each edge of network, in order to control the amount of update received by the edge in each step of optimization. Particularly, our proposed technique referred to as"Smoothed-AND (SAND)-masking", not only validates the agreement in the direction of gradients but also promotes the agreement among their magnitudes to further ensure the discovery of invariances across training domains. SAND-mask is validated over the Domainbed benchmark for domain generalization and significantly improves the state-of-the-art accuracy on the Colored MNIST dataset while providing competitive results on other domain generalization datasets.

2021-06-03

ArXiv (preprint)

arxiv.org

Continual Learning in Deep Networks: an Analysis of the Last Layer

Timothee LESORT

Thomas George

Irina Rish

We study how different output layers in a deep neural network learn and forget in continual learning settings. The following three factors… (see more) can affect catastrophic forgetting in the output layer: (1) weights modifications, (2) interference, and (3) projection drift. In this paper, our goal is to provide more insights into how changing the output layers may address (1) and (2). Some potential solutions to those issues are proposed and evaluated here in several continual learning scenarios. We show that the best-performing type of the output layer depends on the data distribution drifts and/or the amount of data available. In particular, in some cases where a standard linear layer would fail, it turns out that changing parameterization is sufficient in order to achieve a significantly better performance, whithout introducing a continual-learning algorithm and instead using the standard SGD to train a model. Our analysis and results shed light on the dynamics of the output layer in continual learning scenarios, and suggest a way of selecting the best type of output layer for a given scenario.

2021-06-02

ArXiv (preprint)

openreview.net

Enquire One’s Parent and Child Before Decision: Fully Exploit Hierarchical Structure for Self-Supervised Taxonomy Expansion

Suyuchen Wang

Ruihui Zhao

X. T. Chen

Yefeng Zheng

Bang Liu

Taxonomy is a hierarchically structured knowledge graph that plays a crucial role in machine intelligence. The taxonomy expansion task aims … (see more)to find a position for a new term in an existing taxonomy to capture the emerging knowledge in the world and keep the taxonomy dynamically updated. Previous taxonomy expansion solutions neglect valuable information brought by the hierarchical structure and evaluate the correctness of merely an added edge, which downgrade the problem to node-pair scoring or mini-path classification. In this paper, we propose the Hierarchy Expansion Framework (HEF), which fully exploits the hierarchical structure’s properties to maximize the coherence of expanded taxonomy. HEF makes use of taxonomy’s hierarchical structure in multiple aspects: i) HEF utilizes subtrees containing most relevant nodes as self-supervision data for a complete comparison of parental and sibling relations; ii) HEF adopts a coherence modeling module to evaluate the coherence of a taxonomy’s subtree by integrating hypernymy relation detection and several tree-exclusive features; iii) HEF introduces the Fitting Score for position selection, which explicitly evaluates both path and level selections and takes full advantage of parental relations to interchange information for disambiguation and self-correction. Extensive experiments show that by better exploiting the hierarchical structure and optimizing taxonomy’s coherence, HEF vastly surpasses the prior state-of-the-art on three benchmark datasets by an average improvement of 46.7% in accuracy and 32.3% in mean reciprocal rank.

2021-06-02

Proceedings of the Web Conference 2021 (published)

doi.org

arxiv.org

Brainhack: Developing a culture of open, inclusive, community-driven neuroscience

Rémi Gau

Stephanie Noble

Katja Heuer

Katherine L. Bottenhorn

Isil Poyraz Bilgin

Yufang Yang

Julia M. Huntenburg

Johanna Bayer

Richard A. I. Bethlehem

Shawn A Rhoads

Christoph Vogelbacher

Valentina Borghesani

Elizabeth Levitis

Hao-Ting Wang

Sofie Van Den Bossche

Xenia Kobeleva

Jon Haitz Legarreta

Samuel Guay

Melvin Selim Atay

Gael P. Varoquaux … (see 80 more)

Dorien Huijser

Malin Sandström

Peer Herholz

Samuel A. Nastase

AmanPreet Badhwar

Guillaume Dumas

Simon Schwab

Stefano Moia

Michael Dayan

Yasmine Bassil

Paula P. Brooks

Matteo Mancini

James M. Shine

David O’Connor

Xihe Xie

Davide Poggiali

Patrick Friedrich

Anibal Sólon Heinsfeld

Lydia Riedl

Roberto Toro

César Caballero‐Gaudes

Anders Eklund

Kelly Garner

Christopher Nolan

Damion V. Demeter

Fernando A. Barrios

Junaid S. Merchant

Elizabeth A. McDevitt

Robert Oostenveld

R. Cameron Craddock

Ariel Rokem

Andrew Doyle

Satrajit Ghosh

Aki Nikolaidis

Olivia W. Stanley

Eneko Uruñuela

Nasim Anousheh

Aurina Arnatkevičiūtė

Guillaume Auzias

Dipankar Bachar

Élise Bannier

Ruggero Basanisi

Arshitha Basavaraj

Marco Bedini

Pierre Bellec

R. Austin Benn

Kathryn Berluti

Steffen Bollmann

Saskia Bollmann

Claire Bradley

Jesse A. Brown

Augusto Buchweitz

Patrick Callahan

Micaela Y. Chan

Bramsh Q. Chandio

Theresa W Cheng

Sidhant Chopra

Ai Wern Chung

Thomas Close

Etienne Combrisson

Giorgia Cona

R. Todd Constable

Claire Cury

Kamalaker Dadi

Pablo F. Damasceno

Samir Das

Fabrizio De Vico Fallani

Krista DeStasio

Erin W. Dickie

Lena Dorfschmidt

Eugene Duff

Elizabeth DuPré

Sarah L. Dziura

Nathália Bianchini Esper

Oscar Estéban

Shreyas Fadnavis

Guillaume Flandin

Jessica Flannery

John C. Flournoy

Stephanie J. Forkel

Brainhack is an innovative meeting format that promotes scientific collaboration and education in an open and inclusive environment. Depa… (see more)rting from the formats of typical scientific workshops, these events are based on grassroots projects and training, and foster open and reproducible scientific practices. We describe here the multifaceted, lasting benefits of Brainhacks for individual participants, particularly early career researchers. We further highlight the unique contributions that Brainhacks can make to the research community, contributing to scientific progress by complementing opportunities available in conventional formats.

2021-05-31

Neuron (published)

doi.org

Enabling Technologies for Energy Cloud

Thar Intisar Baker

Zehua Guo

Ali Ismail Ali Awad

Shangguang Wang

Benjamin C. M. Fung

2021-05-31

Journal of Parallel and Distributed Computing (published)

doi.org

ER-AE: Differentially Private Text Generation for Authorship Anonymization

Haohan Bo

Steven H. H. Ding

Benjamin C. M. Fung

Farkhund Iqbal

Most of privacy protection studies for textual data focus on removing explicit sensitive identifiers. However, personal writing style, as a … (see more)strong indicator of the authorship, is often neglected. Recent studies, such as SynTF, have shown promising results on privacy-preserving text mining. However, their anonymization algorithm can only output numeric term vectors which are difficult for the recipients to interpret. We propose a novel text generation model with a two-set exponential mechanism for authorship anonymization. By augmenting the semantic information through a REINFORCE training reward function, the model can generate differentially private text that has a close semantic and similar grammatical structure to the original text while removing personal traits of the writing style. It does not assume any conditioned labels or paralleled text data for training. We evaluate the performance of the proposed model on the real-life peer reviews dataset and the Yelp review dataset. The result suggests that our model outperforms the state-of-the-art on semantic preservation, authorship obfuscation, and stylometric transformation.

2021-05-31

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (published)

doi.org

arxiv.org

Explicitly Modeling Syntax in Language Models with Incremental Parsing and a Dynamic Oracle

Syntax is fundamental to our thinking about language. Failing to capture the structure of input language could lead to generalization proble… (see more)ms and over-parametrization. In the present work, we propose a new syntax-aware language model: Syntactic Ordered Memory (SOM). The model explicitly models the structure with an incremental parser and maintains the conditional probability setting of a standard language model (left-to-right). To train the incremental parser and avoid exposure bias, we also propose a novel dynamic oracle, so that SOM is more robust to wrong parsing decisions. Experiments show that SOM can achieve strong results in language modeling, incremental parsing and syntactic generalization tests, while using fewer parameters than other models.

2021-05-31

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (published)

doi.org

arxiv.org

Imperfect also Deserves Reward: Multi-Level and Sequential Reward Modeling for Better Dialog Management

Zhengxu Hou

Bang Liu

Ruihui Zhao

Zijing Ou

Yafei Liu

Xi Chen 0003

X. T. Chen

Yefeng Zheng

For task-oriented dialog systems, training a Reinforcement Learning (RL) based Dialog Management module suffers from low sample efficiency a… (see more)nd slow convergence speed due to the sparse rewards in RL. To solve this problem, many strategies have been proposed to give proper rewards when training RL, but their rewards lack interpretability and cannot accurately estimate the distribution of state-action pairs in real dialogs. In this paper, we propose a multi-level reward modeling approach that factorizes a reward into a three-level hierarchy: domain, act, and slot. Based on inverse adversarial reinforcement learning, our designed reward model can provide more accurate and explainable reward signals for state-action pairs. Extensive evaluations show that our approach can be applied to a wide range of reinforcement learning-based dialog systems and significantly improves both the performance and the speed of convergence.

2021-05-31

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (published)

doi.org

arxiv.org

Improving Long-Term Metrics in Recommendation Systems using Short-Horizon Reinforcement Learning

Bogdan Mazoure

Paul Mineiro

Pavithra Srinath

Reza Sharifi Sedeh

Doina Precup

Adith Swaminathan

We study session-based recommendation scenarios where we want to recommend items to users during sequential interactions to improve their lo… (see more)ng-term utility. Optimizing a long-term metric is challenging because the learning signal (whether the recommendations achieved their desired goals) is delayed and confounded by other user interactions with the system. Targeting immediately measurable proxies such as clicks can lead to suboptimal recommendations due to misalignment with the long-term metric. We develop a new reinforcement learning algorithm called Short Horizon Policy Improvement (SHPI) that approximates policy-induced drift in user behavior across sessions. SHPI is a straightforward modification of episodic RL algorithms for session-based recommendation, that additionally gives an appropriate termination bonus in each session. Empirical results on four recommendation tasks show that SHPI can outperform state-of-the-art recommendation techniques like matrix factorization with offline proxy signals, bandits with myopic online proxies, and RL baselines with limited amounts of user interaction.

2021-05-31

ArXiv (preprint)

arxiv.org

Modeling Event Plausibility with Consistent Conceptual Abstraction

Ian Porada

Kaheer Suleman

Adam Trischler

Jackie CK Cheung

2021-05-31

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (published)

doi.org

arxiv.org

Data-Efficient Reinforcement Learning with Self-Predictive Representations

Max Schwarzer

Ankesh Anand

Rishab Goel

R Devon Hjelm

Aaron Courville

Philip Bachman

While deep reinforcement learning excels at solving tasks where large amounts of data can be collected through virtually unlimited interacti… (see more)on with the environment, learning from limited interaction remains a key challenge. We posit that an agent can learn more efficiently if we augment reward maximization with self-supervised objectives based on structure in its visual input and sequential interaction with the environment. Our method, Self-Predictive Representations(SPR), trains an agent to predict its own latent state representations multiple steps into the future. We compute target representations for future states using an encoder which is an exponential moving average of the agent's parameters and we make predictions using a learned transition model. On its own, this future prediction objective outperforms prior methods for sample-efficient deep RL from pixels. We further improve performance by adding data augmentation to the future prediction loss, which forces the agent's representations to be consistent across multiple views of an observation. Our full self-supervised objective, which combines future prediction and data augmentation, achieves a median human-normalized score of 0.415 on Atari in a setting limited to 100k steps of environment interaction, which represents a 55% relative improvement over the previous state-of-the-art. Notably, even in this limited data regime, SPR exceeds expert human scores on 7 out of 26 games. The code associated with this work is available at https://github.com/mila-iqia/spr

2021-05-30

Université de Montréal (published)

doi.org

openreview.net

Gotta Go Fast When Generating Data with Score-Based Models

Alexia Jolicoeur-Martineau

Fei Li

Rémi Piché-Taillefer

Tal Kachman

Ioannis Mitliagkas

Score-based (denoising diffusion) generative models have recently gained a lot of success in generating realistic and diverse data. These ap… (see more)proaches define a forward diffusion process for transforming data to noise and generate data by reversing it (thereby going from noise to data). Unfortunately, current score-based models generate data very slowly due to the sheer number of score network evaluations required by numerical SDE solvers. In this work, we aim to accelerate this process by devising a more efficient SDE solver. Existing approaches rely on the Euler-Maruyama (EM) solver, which uses a fixed step size. We found that naively replacing it with other SDE solvers fares poorly - they either result in low-quality samples or become slower than EM. To get around this issue, we carefully devise an SDE solver with adaptive step sizes tailored to score-based generative models piece by piece. Our solver requires only two score function evaluations, rarely rejects samples, and leads to high-quality samples. Our approach generates data 2 to 10 times faster than EM while achieving better or equal sample quality. For high-resolution images, our method leads to significantly higher quality samples than all other methods tested. Our SDE solver has the benefit of requiring no step size tuning.

2021-05-27

ArXiv (preprint)

openreview.net

AI Policy Compass

Venture Scientist Bootcamp

Supervision Requests

Publications

AI Policy Compass

Venture Scientist Bootcamp

Supervision Requests

Popular keywords:

Publications