Publications

Alex Graves

Abdel rahman Mohamed

Adi Haviv

Jonathan Berant

Amir Globerson

Chloe Kiddon

Pedro M. Domingos

Brian Lester

Rami Al-rfou'

Noah Constant. 2021

Pengfei Liu

Weizhe Yuan … (see 6 more)

Jinlan Fu

Zhengbao Jiang

Xiao Liu

Yanan Zheng

Zhengxiao Du

Ming Ding

Pretrained language models (PLMs) have 001 been shown to accumulate factual knowledge 002 from their unsupervised pretraining proce-003 dure… (see more)s (Petroni et al., 2019). Prompting is an 004 effective way to query such knowledge from 005 PLMs. Recently, continuous prompt methods 006 have been shown to have a larger potential 007 than discrete prompt methods in generating ef-008 fective queries (Liu et al., 2021a). However, 009 these methods do not consider symmetry of 010 the task. In this work, we propose Symmet-011 rical Prompt Enhancement (SPE), a continu-012 ous prompt-based method for fact retrieval that 013 leverages the symmetry of the task. Our results 014 on LAMA, a popular fact retrieval dataset, 015 show signiﬁcant improvement of SPE over pre-016 vious prompt methods

2020-12-31

(published)

www.semanticscholar.org

SpeechBrain: A General-Purpose Speech Toolkit

Mirco Ravanelli

Titouan Parcollet

Peter Plantinga

Aku Rouhe

Samuele Cornell

Chien-Feng Liao

Elena Rastorgueva

François Grondin

William Aris

Hwidong Na

Yan Gao

Renato De Mori … (see 1 more)

SpeechBrain is an open-source and all-in-one speech toolkit. It is designed to facilitate the research and development of neural speech proc… (see more)essing technologies by being simple, flexible, user-friendly, and well-documented. This paper describes the core architecture designed to support several tasks of common interest, allowing users to naturally conceive, compare and share novel speech processing pipelines. SpeechBrain achieves competitive or state-of-the-art performance in a wide range of speech benchmarks. It also provides training recipes, pretrained models, and inference scripts for popular speech datasets, as well as tutorials which allow anyone with basic Python proficiency to familiarize themselves with speech technologies.

2020-12-31

arXiv (preprint)

arxiv.org

Systematic Generalisation with Group Invariant Predictions

Faruk Ahmed

Harm van Seijen

Aaron Courville

2020-12-31

ICLR (published)

Tackling Situated Multi-Modal Task-Oriented Dialogs with a Single Transformer Model

−. i.eUT

R´ejean Ducharme

P Vincent

Morgan Kaufmann

Yen-Chun Chen

Linjie Li

Licheng Yu

Matthew Henderson

Blaise Thomson

Ehsan Hosseini-Asl

Bryan McCann

Chien-Sheng Wu

Samuel Humeau

Kurt Shuster

Marie-Anne Lachaux

The Situated Interactive Multi-Modal Conver-001 sations (SIMMC) 2.0 aims to create virtual 002 shopping assistants that can accept complex 0… (see more)03 multi-modal inputs, i.e. visual appearances of 004 objects and user utterances. It consists of four 005 subtasks, multi-modal disambiguation (MM-006 Disamb), multi-modal coreference resolution 007 (MM-Coref), multi-modal dialog state tracking 008 (MM-DST), and response retrieval and genera-009 tion. While many task-oriented dialog systems 010 usually tackle each subtask separately, we pro-011 pose a jointly learned encoder-decoder that per-012 forms all four subtasks at once for efficiency. 013 Moreover, we handle the multi-modality of the 014 challenge by representing visual objects as spe-015 cial tokens whose joint embedding is learned 016 via auxiliary tasks. This approach won the MM-017 Coref and response retrieval subtasks and nom-018 inated runner-up for the remaining subtasks 019 using a single unified model. In particular, 020 our model achieved 81.5% MRR, 71.2% R@1, 021 95.0% R@5, 98.2% R@10, and 1.9 mean rank 022 in response retrieval task, setting a high bar for 023 the state-of-the-art result in the SIMMC 2.0 024 track of the Dialog Systems Technology Chal-025 lenge 10 (DSTC10). 026

2020-12-31

(published)

www.semanticscholar.org

Temporally Abstract Partial Models

Khimya Khetarpal

Zafarali Ahmed

Gheorghe Comanici

Doina Precup

Humans and animals have the ability to reason and make predictions about different courses of action at many time scales. In reinforcement l… (see more)earning, option models (Sutton, Precup \& Singh, 1999; Precup, 2000) provide the framework for this kind of temporally abstract prediction and reasoning. Natural intelligent agents are also able to focus their attention on courses of action that are relevant or feasible in a given situation, sometimes termed affordable actions. In this paper, we define a notion of affordances for options, and develop temporally abstract partial option models, that take into account the fact that an option might be affordable only in certain situations. We analyze the trade-offs between estimation and approximation error in planning and learning when using such models, and identify some interesting special cases. Additionally, we demonstrate empirically the potential impact of partial option models on the efficiency of planning.

2020-12-31

Advances in Neural Information Processing Systems 34 (NeurIPS 2021) (published)

Tensor of Quantitative Equational Theories.

Giorgio Bacci

Radu Mardare

Prakash Panangaden

Gordon Plotkin

2020-12-31

CALCO (published)

Test Sample Accuracy Scales with Training Sample Density in Neural Networks

Xu Ji

Razvan Pascanu

Devon Hjelm

Andrea Vedaldi

Balaji Lakshminarayanan

Intuitively, one would expect accuracy of a trained neural network's prediction on test samples to correlate with how densely the samples ar… (see more)e surrounded by seen training samples in representation space. We find that a bound on empirical training error smoothed across linear activation regions scales inversely with training sample density in representation space. Empirically, we verify this bound is a strong predictor of the inaccuracy of the network's prediction on test samples. For unseen test sets, including those with out-of-distribution samples, ranking test samples by their local region's error bound and discarding samples with the highest bounds raises prediction accuracy by up to 20% in absolute terms for image classification datasets, on average over thresholds.

2020-12-31

arXiv.org (preprint)

Textual Time Travel: A Temporally Informed Approach to Theory of Mind

Akshatha Arodi

Jackie CK Cheung

Natural language processing systems such as dialogue agents should be able to reason about other people’s beliefs, intentions and desires.… (see more) This capability, called theory of mind (ToM), is crucial, as it allows a model to predict and interpret the needs of users based on their mental states. A recent line of research evaluates the ToM capability of existing memoryaugmented neural models through questionanswering. These models perform poorly on false belief tasks where beliefs differ from reality, especially when the dataset contains distracting sentences. In this paper, we propose a new temporally informed approach for improving the ToM capability of memory-augmented neural models. Our model incorporates priors about the entities’ minds and tracks their mental states as they evolve over time through an extended passage. It then responds to queries through textual time travel—i.e., by accessing the stored memory of an earlier time step. We evaluate our model on ToM datasets and find that this approach improves performance, particularly by correcting the predicted mental states to match the false belief.

2020-12-31

Conference on Empirical Methods in Natural Language Processing (published)

On the Expressivity of Markov Reward

David Abel

Will Dabney

Anna Harutyunyan

Mark K. Ho

Michael L. Littman

Doina Precup

Satinder Singh

Reward is the driving force for reinforcement-learning agents. This paper is dedicated to understanding the expressivity of reward as a way … (see more)to capture tasks that we would want an agent to perform. We frame this study around three new abstract notions of"task"that might be desirable: (1) a set of acceptable behaviors, (2) a partial ordering over behaviors, or (3) a partial ordering over trajectories. Our main results prove that while reward can express many of these tasks, there exist instances of each task type that no Markov reward function can capture. We then provide a set of polynomial-time algorithms that construct a Markov reward function that allows an agent to optimize tasks of each of these three types, and correctly determine when no such reward function exists. We conclude with an empirical study that corroborates and illustrates our theoretical findings.

2020-12-31

Advances in Neural Information Processing Systems 34 (NeurIPS 2021) (published)

The Locomotive Assignment Problem with Distributed Power at the Canadian National Railway Company

Camilo Ortiz-Astorquiza

Jean-François Cordeau

Emma Frejinger

Some of the most important optimization problems faced by railway operators arise from the management of their locomotive fleet. In this pap… (see more)er, we study a general version of the locomotive assignment problem encountered at the tactical level by one of the largest railroads in North America: the Canadian National Railway Company (CN). We present a modeling framework with two integer linear programming formulations and contribute to the state of the art by allowing to decide each train's operating mode (distributed power or not) over the whole (weekly) planning horizon without partitioning it into smaller time windows. Given the difficulty to solve the problem, one of the formulations is enhanced through various refinements such as constraint relaxations, preprocessing and fixed cost approximations. We thus achieve a significant reduction in the required computational time to solve instances of realistic size. We also present two versions of a Benders decomposition-based algorithm to obtain feasible solutions. On average, it allows to reduce the associated computational time by two hours. Results from an extensive computational study and a case study with data provided by CN confirm the potential benefits of the model and solution approach.

2020-12-31

Transportation Science (published)

The Machine Learning for Combinatorial Optimization Competition (ML4CO): Results and Insights

Maxime Gasse

Simon Bowly

Quentin Cappart

Jonas Charfreitag

Laurent Charlin

Didier Chételat

Antonia Chmiela

Justin Dumouchelle

Ambros Gleixner

Aleksandr M. Kazachkov

Elias Khalil

Pawel Lichocki

Andrea Lodi

Miles Lubin

Chris J. Maddison

Christopher Morris

Dimitri J. Papageorgiou

Augustin Parjadis

Sebastian Pokutta

Antoine Prouvost … (see 22 more)

Lara Scavuzzo

Giulia Zarpellon

Linxin Yang

Sha Lai

Akang Wang

Xiaodong Luo

Shuchang Zhou

Haohan Huang

Shengcheng Shao

Yuanming Zhu

Akang Wang

Mao Kun

Zixuan Cao

Yuanming Zhu

Zhewei Huang

Shuchang Zhou

C. Binbin

He Minggui

Hao Hao

Shuchang Zhou

Mao Kun

Combinatorial optimization is a well-established area in operations research and computer science. Until recently, its methods have focused … (see more)on solving problem instances in isolation, ignoring that they often stem from related data distributions in practice. However, recent years have seen a surge of interest in using machine learning as a new approach for solving combinatorial problems, either directly as solvers or by enhancing exact solvers. Based on this context, the ML4CO aims at improving state-of-the-art combinatorial optimization solvers by replacing key heuristic components. The competition featured three challenging tasks: finding the best feasible solution, producing the tightest optimality certificate, and giving an appropriate solver configuration. Three realistic datasets were considered: balanced item placement, workload apportionment, and maritime inventory routing. This last dataset was kept anonymous for the contestants.

2020-12-31

NeurIPS (Competition and Demos) (published)

proceedings.mlr.press

The role of case importation in explaining differences in early SARS-CoV-2 transmission dynamics in Canada—A mathematical modeling study of surveillance data

Arnaud Godin

Yiqing Xia

David L Buckeridge

Sharmistha Mishra

Dirk Douwes-Schultz

Yannan Shen

Maxime Lavigne

Mélanie Drolet

Alexandra M Schmidt

Marc Brisson

Mathieu Maheu-Giroux

2020-12-31

International Journal of Infectious Diseases (published)