Jackie Cheung

antoine.dangeard@mila.quebec

Antoine Dangeard

Stagiaire de recherche - McGill University

cesare.spinoso@mila.quebec

Aylin Erman

Maîtrise recherche - McGill University

Co-superviseur⋅e :

Dan Poenaru

aylin.erman@mila.quebec

Caleb Moses

Doctorat - McGill University

caleb.moses@mila.quebec

Cesare Spinoso-Di Piano

Doctorat - McGill University

Bonaventure Dossou

Doctorat - McGill University

bonaventure.dossou@mila.quebec

Google Scholar

Haowei Qiu

Maîtrise professionnelle - McGill University

haowei.qiu@mila.quebec

Ian Porada

Doctorat - McGill University

Ines Arous

Postdoctorat - McGill University

ines.arous@mila.quebec

Google Scholar

Jules Gagnon-marchand

Maîtrise recherche - McGill University

gagnonju@mila.quebec

maxime.darrin@mila.quebec

Kushal Arora

Doctorat - McGill University

Co-superviseur⋅e :

Maîtrise recherche - McGill University

martin.pomsl@mila.quebec

Doctorat - McGill University

Co-superviseur⋅e :

Pablo Piantanida

Doctorat - McGill University

Doctorat - McGill University

michael.runningwolf@mila.quebec

Nathan Zeweniuk

Stagiaire de recherche - McGill University University

nathan.zeweniuk@mila.quebec

Ori Ernst

Postdoctorat - McGill University

ori.ernst@mila.quebec

Postdoctorat - McGill University

rahul.aralikatte@mila.quebec

Stagiaire de recherche - McGill University

steven.koniaev@mila.quebec

Xiyuan Zou

Stagiaire de recherche - McGill University

xiyuan.zou@mila.quebec

Yu Bai

Stagiaire de recherche - McGill University

Yu Lu Liu

Maîtrise recherche - McGill University

yu-lu.liu@mila.quebec

Zichao Li

Doctorat - McGill University

Superviseur⋅e principal⋅e :

Siva Reddy

zichao.li@mila.quebec

Ziling Cheng

Stagiaire de recherche - McGill University

ziling.cheng@mila.quebec

Publications

TeMP: Temporal Message Passing for Temporal Knowledge Graph Completion

Jiapeng Wu

Meng Cao

William Hamilton

Inferring missing facts in temporal knowledge graphs (TKGs) is a fundamental and challenging task. Previous works have approached this probl… (voir plus)em by augmenting methods for static knowledge graphs to leverage time-dependent representations. However, these methods do not explicitly leverage multi-hop structural information and temporal facts from recent time steps to enhance their predictions. Additionally, prior work does not explicitly address the temporal sparsity and variability of entity distributions in TKGs. We propose the Temporal Message Passing (TeMP) framework to address these challenges by combining graph neural networks, temporal dynamics models, data imputation and frequency-based gating techniques. Experiments on standard TKG tasks show that our approach provides substantial gains compared to the previous state of the art, achieving a 10.7% average relative improvement in Hits@10 across three standard benchmarks. Our analysis also reveals important sources of variability both within and across TKG datasets, and we introduce several simple but strong baselines that outperform the prior state of the art in certain settings.

2020-11-01

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) (publié)

TESA: A Task in Entity Semantic Aggregation for Abstractive Summarization

Clément Jumel

Annie Priyadarshini Louis

Human-written texts contain frequent generalizations and semantic aggregation of content. In a document, they may refer to a pair of named e… (voir plus)ntities such as ‘London’ and ‘Paris’ with different expressions: “the major cities”, “the capital cities” and “two European cities”. Yet generation, especially, abstractive summarization systems have so far focused heavily on paraphrasing and simplifying the source content, to the exclusion of such semantic abstraction capabilities. In this paper, we present a new dataset and task aimed at the semantic aggregation of entities. TESA contains a dataset of 5.3K crowd-sourced entity aggregations of Person, Organization, and Location named entities. The aggregations are document-appropriate, meaning that they are produced by annotators to match the situational context of a given news article from the New York Times. We then build baseline models for generating aggregations given a tuple of entities and document context. We finetune on TESA an encoder-decoder language model and compare it with simpler classification methods based on linguistically informed features. Our quantitative and qualitative evaluations show reasonable performance in making a choice from a given list of expressions, but free-form expressions are understandably harder to generate and evaluate.

2020-11-01

Conference on Empirical Methods in Natural Language Processing (publié)

HipoRank: Incorporating Hierarchical and Positional Information into Graph-based Unsupervised Long Document Extractive Summarization

Yue Dong

Andrei Mircea

We propose a novel graph-based ranking model for unsupervised extractive summarization of long documents. Graph-based ranking models typical… (voir plus)ly represent documents as undirected fully-connected graphs, where a node is a sentence, an edge is weighted based on sentence-pair similarity, and sentence importance is measured via node centrality. Our method leverages positional and hierarchical information grounded in discourse structure to augment a document's graph representation with hierarchy and directionality. Experimental results on PubMed and arXiv datasets show that our approach outperforms strong unsupervised baselines by wide margins and performs comparably to some of the state-of-the-art supervised models that are trained on hundreds of thousands of examples. In addition, we find that our method provides comparable improvements with various distributional sentence representations; including BERT and RoBERTa models fine-tuned on sentence similarity.

2020-05-01

ArXiv (prépublication)

Investigating the Influence of Selected Linguistic Features on Authorship Attribution using German News Articles

Manuel Sage

Pietro Cruciata

Raed Abdo

Yaoyao Fiona Zhao

In this work, we perform authorship attri-bution on a new dataset of German news articles. We seek to classify over 3,700 articles to their … (voir plus)ﬁve corresponding authors, using four conventional machine learning approaches (na¨ıve Bayes, logistic regression, SVM and kNN) and a convolutional neural network. We analyze the effect of character and word n-grams on the prediction accuracy, as well as the inﬂuence of stop words, punctuation, numbers, and lowercasing when preprocessing raw text. The experiments show that higher order character n-grams (n = 5,6) perform better than lower orders and word n-grams slightly outperform those with characters. Combining both in fusion models further improves results up to 92% for SVM. A multilayer convolutional structure allows the CNN to achieve 90.5% accuracy. We found stop words and punctuation to be important features for author identiﬁcation; removing them leads to a measurable decrease in performance. Finally, we evaluate the topic dependency of the algorithms by gradually replacing named entities, nouns, verbs and eventually all to-kens in the dataset according to their POS-tags.

2020-01-01

SwissText/KONVENS (publié)

dblp.uni-trier.de

On the Systematicity of Probing Contextualized Word Representations: The Case of Hypernymy in BERT.

Abhilasha Ravichander

Eduard Hovy

Kaheer Suleman

Adam Trischler

2020-01-01

*SEM@COLING (publié)

dblp.uni-trier.de

On Variational Learning of Controllable Representations for Text without Supervision

Peng Xu

Yanshuai Cao

The variational autoencoder (VAE) can learn the manifold of natural images on certain datasets, as evidenced by meaningful interpolating or … (voir plus)extrapolating in the continuous latent space. However, on discrete data such as text, it is unclear if unsupervised learning can discover similar latent space that allows controllable manipulation. In this work, we find that sequence VAEs trained on text fail to properly decode when the latent codes are manipulated, because the modified codes often land in holes or vacant regions in the aggregated posterior latent space, where the decoding network fails to generalize. Both as a validation of the explanation and as a fix to the problem, we propose to constrain the posterior mean to a learned probability simplex, and performs manipulation within this simplex. Our proposed method mitigates the latent vacancy problem and achieves the first success in unsupervised learning of controllable representations for text. Empirically, our method outperforms unsupervised baselines and strong supervised approaches on text style transfer, and is capable of performing more flexible fine-grained control over text generation than existing methods.

2020-01-01

ICML (publié)

proceedings.mlr.press

openreview.net

Deconstructing and reconstructing word embedding algorithms

Edward Daniel Newell

Kian Kenyon-Dean

Uncontextualized word embeddings are reliable feature representations of words used to obtain high quality results for various NLP applicati… (voir plus)ons. Given the historical success of word embeddings in NLP, we propose a retrospective on some of the most well-known word embedding algorithms. In this work, we deconstruct Word2vec, GloVe, and others, into a common form, unveiling some of the necessary and sufficient conditions required for making performant word embeddings. We find that each algorithm: (1) fits vector-covector dot products to approximate pointwise mutual information (PMI); and, (2) modulates the loss gradient to balance weak and strong signals. We demonstrate that these two algorithmic features are sufficient conditions to construct a novel word embedding algorithm, Hilbert-MLE. We find that its embeddings obtain equivalent or better performance against other algorithms across 17 intrinsic and extrinsic datasets.

2019-11-29

ArXiv (prépublication)

Preventing Posterior Collapse in Sequence VAEs with Pooling

Teng Long

Yanshuai Cao

Variational Autoencoders (VAEs) hold great potential for modelling text, as they could in theory separate high-level semantic and syntactic … (voir plus)properties from local regularities of natural language. Practically, however, VAEs with autoregressive decoders often suffer from posterior collapse, a phenomenon where the model learns to ignore the latent variables, causing the sequence VAE to degenerate into a language model. Previous works attempt to solve this problem with complex architectural changes or costly optimization schemes. In this paper, we argue that posterior collapse is caused in part by the encoder network failing to capture the input variabilities. We verify this hypothesis empirically and propose a straightforward fix using pooling. This simple technique effectively prevents posterior collapse, allowing the model to achieve significantly better data log-likelihood than standard sequence VAEs. Compared to the previous SOTA on preventing posterior collapse, we are able to achieve comparable performances while being significantly faster.

2019-11-10

ArXiv (prépublication)

Can a Gorilla Ride a Camel? Learning Semantic Plausibility from Text

Ian Porada

Kaheer Suleman

2019-11-01

Proceedings of the First Workshop on Commonsense Inference in Natural Language Processing (publié)

Countering the Effects of Lead Bias in News Summarization via Multi-Stage Training and Auxiliary Losses

Matt Grenander

Yue Dong

Annie Priyadarshini Louis

Sentence position is a strong feature for news summarization, since the lead often (but not always) summarizes the key points of the article… (voir plus). In this paper, we show that recent neural systems excessively exploit this trend, which although powerful for many inputs, is also detrimental when summarizing documents where important content should be extracted from later parts of the article. We propose two techniques to make systems sensitive to the importance of content in different parts of the article. The first technique employs ‘unbiased’ data; i.e., randomly shuffled sentences of the source document, to pretrain the model. The second technique uses an auxiliary ROUGE-based loss that encourages the model to distribute importance scores throughout a document by mimicking sentence-level ROUGE scores on the training data. We show that these techniques significantly improve the performance of a competitive reinforcement learning based extractive system, with the auxiliary loss being more powerful than pretraining.

2019-11-01

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) (publié)

How Reasonable are Common-Sense Reasoning Tasks: A Case-Study on the Winograd Schema Challenge and SWAG

Paul Trichelair

Ali Emami

Adam Trischler

Kaheer Suleman

Recent studies have significantly improved the state-of-the-art on common-sense reasoning (CSR) benchmarks like the Winograd Schema Challeng… (voir plus)e (WSC) and SWAG. The question we ask in this paper is whether improved performance on these benchmarks represents genuine progress towards common-sense-enabled systems. We make case studies of both benchmarks and design protocols that clarify and qualify the results of previous work by analyzing threats to the validity of previous experimental designs. Our protocols account for several properties prevalent in common-sense benchmarks including size limitations, structural regularities, and variable instance difficulty.

2019-11-01

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) (publié)

Referring Expression Generation Using Entity Profiles

Meng Cao

2019-11-01

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) (publié)