Publications

BanditSum: Extractive Summarization as a Contextual Bandit
Yue Dong
Yikang Shen
Eric Crawford
Herke van Hoof
Jackie Cheung
In this work, we propose a novel method for training neural networks to perform single-document extractive summarization without heuristical… (see more)ly-generated extractive labels. We call our approach BanditSum as it treats extractive summarization as a contextual bandit (CB) problem, where the model receives a document to summarize (the context), and chooses a sequence of sentences to include in the summary (the action). A policy gradient reinforcement learning algorithm is used to train the model to select sequences of sentences that maximize ROUGE score. We perform a series of experiments demonstrating that BanditSum is able to achieve ROUGE scores that are better than or comparable to the state-of-the-art for extractive summarization, and converges using significantly fewer update steps than competing approaches. In addition, we show empirically that BanditSum performs significantly better than competing approaches when good summary sentences appear late in the source document.
2018-10-01
Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (published)
doi.org
arxiv.org
HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering
Zhilin Yang
Peng Qi
Saizheng Zhang
Yoshua Bengio
William W. Cohen
Russ Salakhutdinov
Christopher D Manning
Existing question answering (QA) datasets fail to train QA systems to perform complex reasoning and provide explanations for answers. We int… (see more)roduce HotpotQA, a new dataset with 113k Wikipedia-based question-answer pairs with four key features: (1) the questions require finding and reasoning over multiple supporting documents to answer; (2) the questions are diverse and not constrained to any pre-existing knowledge bases or knowledge schemas; (3) we provide sentence-level supporting facts required for reasoning, allowing QA systems to reason with strong supervision and explain the predictions; (4) we offer a new type of factoid comparison questions to test QA systems’ ability to extract relevant facts and perform necessary comparison. We show that HotpotQA is challenging for the latest QA systems, and the supporting facts enable models to improve performance and make explainable predictions.
2018-10-01
Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (published)
doi.org
arxiv.org
A Knowledge Hunting Framework for Common Sense Reasoning
Ali Emami
Noelia De La Cruz
Adam Trischler
Kaheer Suleman
Jackie Cheung
We introduce an automatic system that achieves state-of-the-art results on the Winograd Schema Challenge (WSC), a common sense reasoning tas… (see more)k that requires diverse, complex forms of inference and knowledge. Our method uses a knowledge hunting module to gather text from the web, which serves as evidence for candidate problem resolutions. Given an input problem, our system generates relevant queries to send to a search engine, then extracts and classifies knowledge from the returned results and weighs them to make a resolution. Our approach improves F1 performance on the full WSC by 0.21 over the previous best and represents the first system to exceed 0.5 F1. We further demonstrate that the approach is competitive on the Choice of Plausible Alternatives (COPA) task, which suggests that it is generally applicable.
2018-10-01
Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (published)
doi.org
arxiv.org
Introduction to NIPS 2017 Competition Track
Sergio Escalera
Markus Weimer
Mikhail Burtsev
Valentin Malykh
Varvara Logacheva
Ryan Lowe
Iulian V. Serban
Yoshua Bengio
Alexander Rudnicky
Alan W. Black
Shrimai Prabhumoye
Łukasz Kidziński
Sharada Prasanna Mohanty
Carmichael F. Ong
Jennifer L. Hicks
Sergey Levine
Marcel Salathé
Scott Delp
Iker Huerga
Alexander Grigorenko … (see 19 more)
Leifur Thorbergsson
Anasuya Das
Kyla Nemitz
Jenna Sandker
Stephen King
Alexander S. Ecker
Leon A. Gatys
Matthias Bethge
Jordan Boyd-Graber
Shi Feng
Pedro Rodriguez
Mohit Iyyer
He He
Hal Daumé III
Sean McGregor
Amir Banifatemi
Alexey Kurakin
Ian G Goodfellow
Samy Bengio
2018-09-28
The NIPS '17 Competition: Building Intelligent Systems (published)
doi.org
The First Conversational Intelligence Challenge
Mikhail Burtsev
Varvara Logacheva
Valentin Malykh
Iulian V. Serban
Ryan Lowe
Shrimai Prabhumoye
Alan W. Black
Alexander Rudnicky
Yoshua Bengio
2018-09-28
The NIPS '17 Competition: Building Intelligent Systems (published)
doi.org
Deep Graph Infomax
Petar Veličković
William Fedus
William L. Hamilton
Pietro Lio
Yoshua Bengio
(Rex) Devon Hjelm
We present Deep Graph Infomax (DGI), a general approach for learning node representations within graph-structured data in an unsupervised ma… (see more)nner. DGI relies on maximizing mutual information between patch representations and corresponding high-level summaries of graphs---both derived using established graph convolutional network architectures. The learnt patch representations summarize subgraphs centered around nodes of interest, and can thus be reused for downstream node-wise learning tasks. In contrast to most prior approaches to unsupervised learning with GCNs, DGI does not rely on random walk objectives, and is readily applicable to both transductive and inductive learning setups. We demonstrate competitive performance on a variety of node classification benchmarks, which at times even exceeds the performance of supervised learning.
2018-09-27
ArXiv (preprint)
arxiv.org
arxiv.org
Deep Graph Infomax
Petar Veličković
William Fedus
William L. Hamilton
Pietro Lio
Yoshua Bengio
(Rex) Devon Hjelm
We present Deep Graph Infomax (DGI), a general approach for learning node representations within graph-structured data in an unsupervised ma… (see more)nner. DGI relies on maximizing mutual information between patch representations and corresponding high-level summaries of graphs---both derived using established graph convolutional network architectures. The learnt patch representations summarize subgraphs centered around nodes of interest, and can thus be reused for downstream node-wise learning tasks. In contrast to most prior approaches to unsupervised learning with GCNs, DGI does not rely on random walk objectives, and is readily applicable to both transductive and inductive learning setups. We demonstrate competitive performance on a variety of node classification benchmarks, which at times even exceeds the performance of supervised learning.
2018-09-27
ArXiv (preprint)
arxiv.org
arxiv.org
Deep Graph Infomax
Petar Veličković
William Fedus
William L. Hamilton
Pietro Lio
Yoshua Bengio
(Rex) Devon Hjelm
We present Deep Graph Infomax (DGI), a general approach for learning node representations within graph-structured data in an unsupervised ma… (see more)nner. DGI relies on maximizing mutual information between patch representations and corresponding high-level summaries of graphs---both derived using established graph convolutional network architectures. The learnt patch representations summarize subgraphs centered around nodes of interest, and can thus be reused for downstream node-wise learning tasks. In contrast to most prior approaches to unsupervised learning with GCNs, DGI does not rely on random walk objectives, and is readily applicable to both transductive and inductive learning setups. We demonstrate competitive performance on a variety of node classification benchmarks, which at times even exceeds the performance of supervised learning.
2018-09-27
ArXiv (preprint)
arxiv.org
arxiv.org
Deep Graph Infomax
Petar Veličković
William Fedus
William L. Hamilton
Pietro Lio
Yoshua Bengio
(Rex) Devon Hjelm
We present Deep Graph Infomax (DGI), a general approach for learning node representations within graph-structured data in an unsupervised ma… (see more)nner. DGI relies on maximizing mutual information between patch representations and corresponding high-level summaries of graphs---both derived using established graph convolutional network architectures. The learnt patch representations summarize subgraphs centered around nodes of interest, and can thus be reused for downstream node-wise learning tasks. In contrast to most prior approaches to unsupervised learning with GCNs, DGI does not rely on random walk objectives, and is readily applicable to both transductive and inductive learning setups. We demonstrate competitive performance on a variety of node classification benchmarks, which at times even exceeds the performance of supervised learning.
2018-09-27
ArXiv (preprint)
arxiv.org
arxiv.org
Deep Graph Infomax
Petar Veličković
William Fedus
William L. Hamilton
Pietro Lio
Yoshua Bengio
(Rex) Devon Hjelm
2018-09-27
ArXiv (preprint)
arxiv.org
arxiv.org
Deep Graph Infomax
Petar Veličković
William Fedus
William L. Hamilton
Pietro Lio
Yoshua Bengio
(Rex) Devon Hjelm
We present Deep Graph Infomax (DGI), a general approach for learning node representations within graph-structured data in an unsupervised ma… (see more)nner. DGI relies on maximizing mutual information between patch representations and corresponding high-level summaries of graphs---both derived using established graph convolutional network architectures. The learnt patch representations summarize subgraphs centered around nodes of interest, and can thus be reused for downstream node-wise learning tasks. In contrast to most prior approaches to unsupervised learning with GCNs, DGI does not rely on random walk objectives, and is readily applicable to both transductive and inductive learning setups. We demonstrate competitive performance on a variety of node classification benchmarks, which at times even exceeds the performance of supervised learning.
2018-09-27
ArXiv (preprint)
arxiv.org
arxiv.org
Deep Graph Infomax
Petar Veličković
William Fedus
William L. Hamilton
Pietro Lio
Yoshua Bengio
(Rex) Devon Hjelm
2018-09-27
ArXiv (preprint)
arxiv.org
arxiv.org