Jie Fu

Stefano Ermon

Doina Precup

2023-11-18

logconference.io/LOG/2023/Conference (poster)

Think Before You Act: Decision Transformers with Internal Working Memory

Jikun Kang

Romain Laroche

Xingdi Yuan

Adam P. Trischler

Xuefei Liu

Large language model (LLM)-based decision-making agents have shown the ability to generalize across multiple tasks. However, their performan… (see more)ce relies on massive data and compute. We argue that this inefficiency stems from the forgetting phenomenon, in which a model memorizes its behaviors in parameters throughout training. As a result, training on a new task may deteriorate the model's performance on previous tasks. In contrast to LLMs' implicit memory mechanism, the human brain utilizes distributed memory storage, which helps manage and organize multiple skills efficiently, mitigating the forgetting phenomenon. Thus inspired, we propose an internal working memory module to store, blend, and retrieve information for different downstream tasks. Evaluation results show that the proposed method improves training efficiency and generalization in both Atari games and meta-world object manipulation tasks. Moreover, we demonstrate that memory fine-tuning further enhances the adaptability of the proposed architecture.

2023-05-24

ArXiv (preprint)

Think Before You Act: Decision Transformers with Internal Working Memory

Jikun Kang

Romain Laroche

Xingdi Yuan

Adam Trischler

Xue (Steve) Liu

Large language model (LLM)-based decision-making agents have shown the ability to generalize across multiple tasks. However, their performan… (see more)ce relies on massive data and compute. We argue that this inefficiency stems from the forgetting phenomenon, in which a model memorizes its behaviors in parameters throughout training. As a result, training on a new task may deteriorate the model's performance on previous tasks. In contrast to LLMs' implicit memory mechanism, the human brain utilizes distributed memory storage, which helps manage and organize multiple skills efficiently, mitigating the forgetting phenomenon. Thus inspired, we propose an internal working memory module to store, blend, and retrieve information for different downstream tasks. Evaluation results show that the proposed method improves training efficiency and generalization in both Atari games and meta-world object manipulation tasks. Moreover, we demonstrate that memory fine-tuning further enhances the adaptability of the proposed architecture.

2023-05-24

ArXiv (preprint)

When Do Graph Neural Networks Help with Node Classification? Investigating the Impact of Homophily Principle on Node Distinguishability

Sitao Luan

Chenqing Hua

Minkai Xu

Qincheng Lu

Jiaqi Zhu

Xiao-Wen Chang

Jure Leskovec

Doina Precup

2023-04-25

ArXiv (preprint)

Learning Multi-Objective Curricula for Robotic Policy Learning

Jikun Kang

Miao Liu

2023-03-06

Proceedings of The 6th Conference on Robot Learning (published)

proceedings.mlr.press

When Do Graph Neural Networks Help with Node Classification: Investigating the Homophily Principle on Node Distinguishability

Sitao Luan

Chenqing Hua

Minkai Xu

Qincheng Lu

Jiaqi Zhu

Xiao-Wen Chang

Jure Leskovec

Doina Precup

Homophily principle, i.e., nodes with the same labels are more likely to be connected, was believed to be the main reason for the performanc… (see more)e superiority of Graph Neural Networks (GNNs) over Neural Networks (NNs) on Node Classiﬁcation (NC) tasks. Recently, people have developed theoretical results arguing that, even though the homophily principle is broken, the advantage of GNNs can still hold as long as nodes from the same class share similar neighborhood patterns [29], which questions the validity of homophily. However, this argument only considers intra-class Node Distinguishability (ND) and ignores inter-class ND, which is insufﬁcient to study the effect of homophily. In this paper, we ﬁrst demonstrate the aforementioned insufﬁciency with examples and argue that an ideal situation for ND is to have smaller intra-class ND than inter-class ND. To formulate this idea and have a better understanding of homophily, we propose Contextual Stochastic Block Model for Homophily (CSBM-H) and deﬁne two metrics, Probabilistic Bayes Error (PBE) and Expected Negative KL-divergence (ENKL), to quantify ND, through which we can also ﬁnd how intra- and inter-class ND inﬂuence ND together. We visualize the results and give detailed analysis. Through experiments, we veriﬁed that the superiority of GNNs is

Biological Sequence Design with GFlowNets

Moksh J. Jain

Emmanuel Bengio

Alex Hernandez-Garcia

Jarrid Rector-Brooks

Bonaventure F. P. Dossou

Micheal Kilgour

Payel Das

2022-06-28

Proceedings of the 39th International Conference on Machine Learning (published)

Unifying Likelihood-free Inference with Black-box Optimization and Beyond

Black-box optimization formulations for biological sequence design have drawn recent attention due to their promising potential impact on th… (see more)e pharmaceutical industry. In this work, we propose to unify two seemingly distinct worlds: likelihood-free inference and black-box optimization, under one probabilistic framework. In tandem, we provide a recipe for constructing various sequence design methods based on this framework. We show how previous optimization approaches can be"reinvented"in our framework, and further propose new probabilistic black-box optimization algorithms. Extensive experiments on sequence design application illustrate the benefits of the proposed methodology.

2022-01-01

ICLR (published)

FloW: A Dataset and Benchmark for Floating Waste Detection in Inland Waters

Yuwei Cheng

Jiannan Zhu

Mengxin Jiang

Changsong Pang

Peidong Wang

Kris Sankaran

Olawale Moses Onabola

Yimin Liu

Dianbo Liu

Yoshua Bengio

Marine debris is severely threatening the marine lives and causing sustained pollution to the whole ecosystem. To prevent the wastes from ge… (see more)tting into the ocean, it is helpful to clean up the floating wastes in inland waters using the autonomous cleaning devices like unmanned surface vehicles. The cleaning efficiency relies on a high-accurate and robust object detection system. However, the small size of the target, the strong light reflection over water surface, and the reflection of other objects on bank-side all bring challenges to the vision-based object detection system. To promote the practical application for autonomous floating wastes cleaning, we present FloW†, the first dataset for floating waste detection in inland water areas. The dataset consists of an image sub-dataset FloW-Img and a multimodal sub-dataset FloW-RI which contains synchronized millimeter wave radar data and images. Accurate annotations for images and radar data are provided, supporting floating waste detection strategies based on image, radar data, and the fusion of two sensors. We perform several baseline experiments on our dataset, including vision-based and radar-based detection methods. The results show that, the detection accuracy is relatively low and floating waste detection still remains a challenging task.

2021-10-01

IEEE International Conference on Computer Vision (published)

Unifying Likelihood-free Inference with Black-box Sequence Design and Beyond

2021-01-01

arXiv.org (preprint)

dblp.uni-trier.de

Interactive Machine Comprehension with Information Seeking Agents

Xingdi Yuan

Marc-Alexandre Côté

Yi Tay

Chris Pal

Adam Trischler

Existing machine reading comprehension (MRC) models do not scale effectively to real-world applications like web-level information retrieval… (see more) and question answering (QA). We argue that this stems from the nature of MRC datasets: most of these are static environments wherein the supporting documents and all necessary information are fully observed. In this paper, we propose a simple method that reframes existing MRC datasets as interactive, partially observable environments. Specifically, we “occlude” the majority of a document’s text and add context-sensitive commands that reveal “glimpses” of the hidden text to a model. We repurpose SQuAD and NewsQA as an initial case study, and then show how the interactive corpora can be used to train a model that seeks relevant information through sequential decision making. We believe that this setting can contribute in scaling models to web-level QA scenarios.

2020-07-01

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (published)

Would you Rather? A New Benchmark for Learning Machine Alignment with Cultural Values and Social Preferences

Yi Tay

Donovan Ong

Alvin Chan

Nancy Chen

Anh Tuan Luu

Chris Pal

2020-07-01

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (published)