Publications

Variance Reduction is an Antidote to Byzantines: Better Rates, Weaker Assumptions and Communication Compression as a Cherry on the Top

Eduard Gorbunov

Samuel Horváth

Peter Richtárik

Gauthier Gidel

Byzantine-robustness has been gaining a lot of attention due to the growth of the interest in collaborative and federated learning. However,… (see more) many fruitful directions, such as the usage of variance reduction for achieving robustness and communication compression for reducing communication costs, remain weakly explored in the field. This work addresses this gap and proposes Byz-VR-MARINA - a new Byzantine-tolerant method with variance reduction and compression. A key message of our paper is that variance reduction is key to fighting Byzantine workers more effectively. At the same time, communication compression is a bonus that makes the process more communication efficient. We derive theoretical convergence guarantees for Byz-VR-MARINA outperforming previous state-of-the-art for general non-convex and Polyak-Lojasiewicz loss functions. Unlike the concurrent Byzantine-robust methods with variance reduction and/or compression, our complexity results are tight and do not rely on restrictive assumptions such as boundedness of the gradients or limited compression. Moreover, we provide the first analysis of a Byzantine-tolerant method supporting non-uniform sampling of stochastic gradients. Numerical experiments corroborate our theoretical findings.

2022-12-31

International Conference on Learning Representations (published)

Versatile Energy-Based Probabilistic Models for High Energy Physics

Taoli Cheng

Aaron Courville

As a classical generative modeling approach, energy-based models have the natural advantage of flexibility in the form of the energy functio… (see more)n. Recently, energy-based models have achieved great success in modeling high-dimensional data in computer vision and natural language processing. In line with these advancements, we build a multi-purpose energy-based probabilistic model for High Energy Physics events at the Large Hadron Collider. This framework builds on a powerful generative model and describes higher-order inter-particle interactions. It suits different encoding architectures and builds on implicit generation. As for applicative aspects, it can serve as a powerful parameterized event generator for physics simulation, a generic anomalous signal detector free from spurious correlations, and an augmented event classifier for particle identification.

2022-12-31

arXiv.org (preprint)

Video Killed the HD-Map: Predicting Multi-Agent Behavior Directly From Aerial Images

Yunpeng Liu

Vasileios Lioutas

Jonathan Wilder Lavington

Matthew Niedoba

Justice Sefas

Setareh Dabiri

Dylan Green

Xiaoxuan Liang

Berend Zwartsenberg

Adam Ścibior

Frank N. Wood

The development of algorithms that learn multi-agent behavioral models using human demonstrations has led to increasingly realistic simulati… (see more)ons in the field of autonomous driving. In general, such models learn to jointly predict trajectories for all controlled agents by exploiting road context information such as drivable lanes obtained from manually annotated high-definition (HD) maps. Recent studies show that these models can greatly benefit from increasing the amount of human data available for training. However, the manual annotation of HD maps which is necessary for every new location puts a bottleneck on efficiently scaling up human traffic datasets. We propose an aerial image-based map (AIM) representation that requires minimal annotation and provides rich road context information for traffic agents like pedestrians and vehicles. We evaluate multi-agent trajectory prediction using the AIM by incorporating it into a differentiable driving simulator as an image-texture-based differentiable rendering module. Our results demonstrate competitive multi-agent trajectory prediction performance especially for pedestrians in the scene when using our AIM representation as compared to models trained with rasterized HD maps.

2022-12-31

2023 IEEE 26th International Conference on Intelligent Transportation Systems (ITSC) (published)

Willingness to Engage in Shared Decision Making: Impact of an Educational Intervention for Resident Physicians (SDM-FM)

Roland M. Grad

A. Sandhu

Michael Ferrante

Vinita D'souza

Lily Puterman-salzman

S. A. Rahimi

Gabrielle Stevens

G. Elwyn

2022-12-31

Education and training (published)

Workflow Discovery from Dialogues in the Low Data Regime

Amine El hattami

Stefania Raimondo

Issam Hadj Laradji

David Vázquez

Pau Rodríguez

Christopher Pal

Text-based dialogues are now widely used to solve real-world problems. In cases where solution strategies are already known, they can someti… (see more)mes be codified into workflows and used to guide humans or artificial agents through the task of helping clients. We introduce a new problem formulation that we call Workflow Discovery (WD) in which we are interested in the situation where a formal workflow may not yet exist. Still, we wish to discover the set of actions that have been taken to resolve a particular problem. We also examine a sequence-to-sequence (Seq2Seq) approach for this novel task. We present experiments where we extract workflows from dialogues in the Action-Based Conversations Dataset (ABCD). Since the ABCD dialogues follow known workflows to guide agents, we can evaluate our ability to extract such workflows using ground truth sequences of actions. We propose and evaluate an approach that conditions models on the set of possible actions, and we show that using this strategy, we can improve WD performance. Our conditioning approach also improves zero-shot and few-shot WD performance when transferring learned models to unseen domains within and across datasets. Further, on ABCD a modified variant of our Seq2Seq method achieves state-of-the-art performance on related but different problems of Action State Tracking (AST) and Cascading Dialogue Success (CDS) across many evaluation metrics.

2022-12-31

Trans. Mach. Learn. Res. (published)

"Your child needs surgery": A survey-based evaluation of simulated expert consent conversations by key stakeholders.

Zoe Atsaidis

Stephan Robitaille

Elena Guadagno

Jeffrey Wiseman

Sherif Emil

Dan Poenaru

2022-12-31

Journal of Pediatric Surgery (published)

Offline Policy Optimization in RL with Variance Regularizaton

Riashat Islam

Samarth Sinha

Homanga Bharadhwaj

Samin Yeasar Arnob

Zhuoran Yang

Animesh Garg

Zhaoran Wang

Lihong Li

Doina Precup

2022-12-28

ArXiv (preprint)

Simplicity and learning to distinguish arguments from modifiers

Leon Bergen

E. Gibson

Timothy J. O'Donnell

2022-12-27

Journal of Language Modelling (published)

How programmers find online learning resources

Deeksha M. Arya

Jin L.C. Guo

Martin P. Robillard

2022-12-23

Empirical Software Engineering (published)

FaithDial: A Faithful Benchmark for Information-Seeking Dialogue

Nouha Dziri

Ehsan Kamalloo

Sivan Milton

Osmar Zaiane

Mo Yu

Edoardo M. Ponti

Siva Reddy

The goal of information-seeking dialogue is to respond to seeker queries with natural language utterances that are grounded on knowledge sou… (see more)rces. However, dialogue systems often produce unsupported utterances, a phenomenon known as hallucination. To mitigate this behavior, we adopt a data-centric solution and create FaithDial, a new benchmark for hallucination-free dialogues, by editing hallucinated responses in the Wizard of Wikipedia (WoW) benchmark. We observe that FaithDial is more faithful than WoW while also maintaining engaging conversations. We show that FaithDial can serve as training signal for: i) a hallucination critic, which discriminates whether an utterance is faithful or not, and boosts the performance by 12.8 F1 score on the BEGIN benchmark compared to existing datasets for dialogue coherence; ii) high-quality dialogue generation. We benchmark a series of state-of-the-art models and propose an auxiliary contrastive objective that achieves the highest level of faithfulness and abstractiveness based on several automated metrics. Further, we find that the benefits of FaithDial generalize to zero-shot transfer on other datasets, such as CMU-Dog and TopicalChat. Finally, human evaluation reveals that responses generated by models trained on FaithDial are perceived as more interpretable, cooperative, and engaging.

2022-12-22

Transactions of the Association for Computational Linguistics (published)

Post-hoc Interpretability for Neural NLP: A Survey

Andreas Madsen

Siva Reddy

A. Chandar

2022-12-22

ACM Computing Surveys (published)