Jackie Cheung

antoine.dangeard@mila.quebec

Antoine Dangeard

Research Intern - McGill University

cesare.spinoso@mila.quebec

Aylin Erman

Master's Research - McGill University

Co-supervisor :

Dan Poenaru

aylin.erman@mila.quebec

Caleb Moses

PhD - McGill University

caleb.moses@mila.quebec

Cesare Spinoso-Di Piano

PhD - McGill University

Bonaventure Dossou

PhD - McGill University

bonaventure.dossou@mila.quebec

Google Scholar

Haowei Qiu

Professional Master's - McGill University

haowei.qiu@mila.quebec

Ian Porada

PhD - McGill University

Ines Arous

Postdoctorate - McGill University

ines.arous@mila.quebec

Google Scholar

Jules Gagnon-marchand

Master's Research - McGill University

gagnonju@mila.quebec

maxime.darrin@mila.quebec

Kushal Arora

PhD - McGill University

Co-supervisor :

Master's Research - McGill University

martin.pomsl@mila.quebec

PhD - McGill University

Co-supervisor :

Pablo Piantanida

PhD - McGill University

PhD - McGill University

michael.runningwolf@mila.quebec

Nathan Zeweniuk

Research Intern - McGill University University

nathan.zeweniuk@mila.quebec

Ori Ernst

Postdoctorate - McGill University

ori.ernst@mila.quebec

Postdoctorate - McGill University

rahul.aralikatte@mila.quebec

Research Intern - McGill University

steven.koniaev@mila.quebec

Xiyuan Zou

Research Intern - McGill University

xiyuan.zou@mila.quebec

Yu Bai

Research Intern - McGill University

Yu Lu Liu

Master's Research - McGill University

yu-lu.liu@mila.quebec

Zichao Li

PhD - McGill University

Principal supervisor :

Siva Reddy

zichao.li@mila.quebec

Ziling Cheng

Research Intern - McGill University

ziling.cheng@mila.quebec

Publications

Contextualized Non-local Neural Networks for Sequence Learning

Pengfei Liu

Shuaichen Chang

Xuanjing Huang

Jian Tang

Recently, a large number of neural mechanisms and models have been proposed for sequence learning, of which selfattention, as exemplified by… (see more) the Transformer model, and graph neural networks (GNNs) have attracted much attention. In this paper, we propose an approach that combines and draws on the complementary strengths of these two methods. Specifically, we propose contextualized non-local neural networks (CN3), which can both dynamically construct a task-specific structure of a sentence and leverage rich local dependencies within a particular neighbourhood.Experimental results on ten NLP tasks in text classification, semantic matching, and sequence labelling show that our proposed model outperforms competitive baselines and discovers task-specific dependency structures, thus providing better interpretability to users.

2019-07-17

Proceedings of the AAAI Conference on Artificial Intelligence (published)

Generating Character Descriptions for Automatic Summarization of Fiction

Weiwei Zhang

J. Oren

Summaries of fictional stories allow readers to quickly decide whether or not a story catches their interest. A major challenge in automatic… (see more) summarization of fiction is the lack of standardized evaluation methodology or high-quality datasets for experimentation. In this work, we take a bottomup approach to this problem by assuming that story authors are uniquely qualified to inform such decisions. We collect a dataset of one million fiction stories with accompanying author-written summaries from Wattpad, an online story sharing platform. We identify commonly occurring summary components, of which a description of the main characters is the most frequent, and elicit descriptions of main characters directly from the authors for a sample of the stories. We propose two approaches to generate character descriptions, one based on ranking attributes found in the story text, the other based on classifying into a list of pre-defined attributes. We find that the classification-based approach performs the best in predicting character descriptions.

2019-07-17

Proceedings of the AAAI Conference on Artificial Intelligence (published)

Learning Multi-Task Communication with Message Passing for Sequence Learning

Pengfei Liu

Jie Fu

Yue Dong

Xipeng Qiu

We present two architectures for multi-task learning with neural sequence models. Our approach allows the relationships between different ta… (see more)sks to be learned dynamically, rather than using an ad-hoc pre-defined structure as in previous work. We adopt the idea from message-passing graph neural networks, and propose a general graph multi-task learning framework in which different tasks can communicate with each other in an effective and interpretable way. We conduct extensive experiments in text classification and sequence labelling to evaluate our approach on multi-task learning and transfer learning. The empirical results show that our models not only outperform competitive baselines, but also learn interpretable and transferable patterns across tasks.

2019-07-17

Proceedings of the AAAI Conference on Artificial Intelligence (published)

A Cross-Domain Transferable Neural Coherence Model

Peng Xu

H. Saghir

Jin Sung Kang

Teng Long

Avishek Joey Bose

Yanshuai Cao

Coherence is an important aspect of text quality and is crucial for ensuring its readability. One important limitation of existing coherence… (see more) models is that training on one domain does not easily generalize to unseen categories of text. Previous work advocates for generative models for cross-domain generalization, because for discriminative models, the space of incoherent sentence orderings to discriminate against during training is prohibitively large. In this work, we propose a local discriminative neural model with a much smaller negative sampling space that can efficiently learn against incorrect orderings. The proposed coherence model is simple in structure, yet it significantly outperforms previous state-of-art methods on a standard benchmark dataset on the Wall Street Journal corpus, as well as in multiple new challenging settings of transfer to unseen categories of discourse on Wikipedia articles.

2019-07-01

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (published)

EditNTS: An Neural Programmer-Interpreter Model for Sentence Simplification through Explicit Editing

Yue Dong

Zichao Li

Mehdi Rezagholizadeh

We present the first sentence simplification model that learns explicit edit operations (ADD, DELETE, and KEEP) via a neural programmer-inte… (see more)rpreter approach. Most current neural sentence simplification systems are variants of sequence-to-sequence models adopted from machine translation. These methods learn to simplify sentences as a byproduct of the fact that they are trained on complex-simple sentence pairs. By contrast, our neural programmer-interpreter is directly trained to predict explicit edit operations on targeted parts of the input sentence, resembling the way that humans perform simplification and revision. Our model outperforms previous state-of-the-art neural sentence simplification models (without external knowledge) by large margins on three benchmark text simplification corpora in terms of SARI (+0.95 WikiLarge, +1.89 WikiSmall, +1.41 Newsela), and is judged by humans to produce overall better and simpler output sentences.

2019-07-01

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (published)

Understanding the Behaviour of Neural Abstractive Summarizers using Contrastive Examples

Krtin Kumar

Neural abstractive summarizers generate summary texts using a language model conditioned on the input source text, and have recently achieve… (see more)d high ROUGE scores on benchmark summarization datasets. We investigate how they achieve this performance with respect to human-written gold-standard abstracts, and whether the systems are able to understand deeper syntactic and semantic structures. We generate a set of contrastive summaries which are perturbed, deficient versions of human-written summaries, and test whether existing neural summarizers score them more highly than the human-written summaries. We analyze their performance on different datasets and find that these systems fail to understand the source text, in a majority of the cases.

2019-06-01

North American Chapter of the Association for Computational Linguistics (published)

Unsupervised Controllable Text Generation with Global Variation Discovery and Disentanglement

Peng Xu

Yanshuai Cao

Existing controllable text generation systems rely on annotated attributes, which greatly limits their capabilities and applications. In thi… (see more)s work, we make the first successful attempt to use VAEs to achieve controllable text generation without supervision. We do so by decomposing the latent space of the VAE into two parts: one incorporates structural constraints to capture dominant global variations implicitly present in the data, e.g., sentiment or topic; the other is unstructured and is used for the reconstruction of the source sentences. With the enforced structural constraint, the underlying global variations will be discovered and disentangled during the training of the VAE. The structural constraint also provides a natural recipe for mitigating posterior collapse for the structured part, which cannot be fully resolved by the existing techniques. On the task of text style transfer, our unsupervised approach achieves significantly better performance than previous supervised approaches. By showcasing generation with finer-grained control including Cards-Against-Humanity-style topic transitions within a sentence, we demonstrate that our model can perform controlled text generation in a more flexible way than existing methods.

2019-05-28

ArXiv (preprint)

What comes next? Extractive summarization by next-sentence prediction

Jingyun Liu

Annie Priyadarshini Louis

Existing approaches to automatic summarization assume that a length limit for the summary is given, and view content selection as an optimiz… (see more)ation problem to maximize informativeness and minimize redundancy within this budget. This framework ignores the fact that human-written summaries have rich internal structure which can be exploited to train a summarization system. We present NEXTSUM, a novel approach to summarization based on a model that predicts the next sentence to include in the summary using not only the source article, but also the summary produced so far. We show that such a model successfully captures summary-specific discourse moves, and leads to better content selection performance, in addition to automatically predicting how long the target summary should be. We perform experiments on the New York Times Annotated Corpus of summaries, where NEXTSUM outperforms lead and content-model summarization baselines by significant margins. We also show that the lengths of summaries produced by our system correlates with the lengths of the human-written gold standards.

2019-01-12

ArXiv (preprint)

Clustering-Oriented Representation Learning with Attractive-Repulsive Loss

Kian Kenyon-Dean

Andre Cianflone

Lucas Caccia

Guillaume Rabusseau

Doina Precup

The standard loss function used to train neural network classifiers, categorical cross-entropy (CCE), seeks to maximize accuracy on the trai… (see more)ning data; building useful representations is not a necessary byproduct of this objective. In this work, we propose clustering-oriented representation learning (COREL) as an alternative to CCE in the context of a generalized attractive-repulsive loss framework. COREL has the consequence of building latent representations that collectively exhibit the quality of natural clustering within the latent space of the final hidden layer, according to a predefined similarity function. Despite being simple to implement, COREL variants outperform or perform equivalently to CCE in a variety of scenarios, including image and news article classification using both feed-forward and convolutional neural networks. Analysis of the latent spaces created with different similarity functions facilitates insights on the different use cases COREL variants can satisfy, where the Cosine-COREL variant makes a consistently clusterable latent space, while Gaussian-COREL consistently obtains better classification accuracy than CCE.

2018-12-18

ArXiv (preprint)

Multi-task Learning over Graph Structures

Pengfei Liu

Jie Fu

Yue Dong

Xipeng Qiu

We present two architectures for multi-task learning with neural sequence models. Our approach allows the relationships between different ta… (see more)sks to be learned dynamically, rather than using an ad-hoc pre-defined structure as in previous work. We adopt the idea from message-passing graph neural networks and propose a general \textbf{graph multi-task learning} framework in which different tasks can communicate with each other in an effective and interpretable way. We conduct extensive experiments in text classification and sequence labeling to evaluate our approach on multi-task learning and transfer learning. The empirical results show that our models not only outperform competitive baselines but also learn interpretable and transferable patterns across tasks.

2018-11-26

ArXiv (preprint)

On the Evaluation of Common-Sense Reasoning in Natural Language Understanding

Paul Trichelair

Ali Emami

Adam Trischler

Kaheer Suleman

Fernando Diaz

The NLP and ML communities have long been interested in developing models capable of common-sense reasoning, and recent works have significa… (see more)ntly improved the state of the art on benchmarks like the Winograd Schema Challenge (WSC). Despite these advances, the complexity of tasks designed to test common-sense reasoning remains under-analyzed. In this paper, we make a case study of the Winograd Schema Challenge and, based on two new measures of instance-level complexity, design a protocol that both clarifies and qualifies the results of previous work. Our protocol accounts for the WSC's limited size and variable instance difficulty, properties common to other common-sense benchmarks. Accounting for these properties when assessing model results may prevent unjustified conclusions.

2018-11-05

arXiv.org (preprint)

dblp.uni-trier.de

The Hard-CoRe Coreference Corpus: Removing Gender and Number Cues for Difficult Pronominal Anaphora Resolution

Ali Emami

Paul Trichelair

Adam Trischler

Kaheer Suleman

Hannes Schulz

We introduce a new benchmark task for coreference resolution, Hard-CoRe, that targets common-sense reasoning and world knowledge. Previous c… (see more)oreference resolution tasks have been overly vulnerable to systems that simply exploit the number and gender of the antecedents, or have been handcrafted and do not reflect the diversity of sentences in naturally occurring text. With these limitations in mind, we present a resolution task that is both challenging and realistic. We demonstrate that various coreference systems, whether rule-based, feature-rich, graphical, or neural-based, perform at random or slightly above-random on the task, whereas human performance is very strong with high inter-annotator agreement. To explain this performance gap, we show empirically that state-of-the art models often fail to capture context and rely only on the antecedents to make a decision.

2018-11-02

ArXiv (preprint)