Portrait de Jie Fu n'est pas disponible

Jie Fu

Alumni

Publications

Unifying Likelihood-Free Inference with Black-Box Optimization and Beyond
Dinghuai Zhang
Jie Fu
Yoshua Bengio
Aaron Courville
Black-box optimization formulations for biological sequence design have drawn recent attention due to their promising potential impact on th… (voir plus)e pharmaceutical industry. In this work, we propose to unify two seemingly distinct worlds: likelihood-free inference and black-box optimization, under one probabilistic framework. In tandem, we provide a recipe for constructing various sequence design methods based on this framework. We show how previous optimization approaches can be "reinvented" in our framework, and further propose new probabilistic black-box optimization algorithms. Extensive experiments on sequence design application illustrate the benefits of the proposed methodology.
2022-01-27
International Conference on Learning Representations (spotlight)
doi.org
openreview.net
FloW: A Dataset and Benchmark for Floating Waste Detection in Inland Waters
Yuwei Cheng
Jiannan Zhu
Mengxin Jiang
Jie Fu
Changsong Pang
Peidong Wang
Kris Sankaran
Olawale Moses Onabola
Yimin Liu
Dianbo Liu
Yoshua Bengio
Marine debris is severely threatening the marine lives and causing sustained pollution to the whole ecosystem. To prevent the wastes from ge… (voir plus)tting into the ocean, it is helpful to clean up the floating wastes in inland waters using the autonomous cleaning devices like unmanned surface vehicles. The cleaning efficiency relies on a high-accurate and robust object detection system. However, the small size of the target, the strong light reflection over water surface, and the reflection of other objects on bank-side all bring challenges to the vision-based object detection system. To promote the practical application for autonomous floating wastes cleaning, we present FloW†, the first dataset for floating waste detection in inland water areas. The dataset consists of an image sub-dataset FloW-Img and a multimodal sub-dataset FloW-RI which contains synchronized millimeter wave radar data and images. Accurate annotations for images and radar data are provided, supporting floating waste detection strategies based on image, radar data, and the fusion of two sensors. We perform several baseline experiments on our dataset, including vision-based and radar-based detection methods. The results show that, the detection accuracy is relatively low and floating waste detection still remains a challenging task.
2021-10-09
IEEE/CVF International Conference on Computer Vision (publié)
doi.org
Unifying Likelihood-free Inference with Black-box Sequence Design and Beyond
Dinghuai Zhang
Jie Fu
Yoshua Bengio
Aaron Courville
2020-12-31
arXiv.org (prépublication)
dblp.uni-trier.de
Interactive Machine Comprehension with Information Seeking Agents
Xingdi Yuan
Jie Fu
Marc-Alexandre Côté
Yi Tay
Christopher Pal
Adam Trischler
Existing machine reading comprehension (MRC) models do not scale effectively to real-world applications like web-level information retrieval… (voir plus) and question answering (QA). We argue that this stems from the nature of MRC datasets: most of these are static environments wherein the supporting documents and all necessary information are fully observed. In this paper, we propose a simple method that reframes existing MRC datasets as interactive, partially observable environments. Specifically, we “occlude” the majority of a document’s text and add context-sensitive commands that reveal “glimpses” of the hidden text to a model. We repurpose SQuAD and NewsQA as an initial case study, and then show how the interactive corpora can be used to train a model that seeks relevant information through sequential decision making. We believe that this setting can contribute in scaling models to web-level QA scenarios.
2020-06-30
Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (publié)
doi.org
arxiv.org
Would you Rather? A New Benchmark for Learning Machine Alignment with Cultural Values and Social Preferences
Yi Tay
Donovan Ong
Jie Fu
Alvin Chan
Nancy Chen
Anh Tuan Luu
Christopher Pal
2020-06-30
Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (publié)
doi.org
Role-Wise Data Augmentation for Knowledge Distillation
Jie Fu
Xue Geng
Zhijian Duan
Bohan Zhuang
Xingdi Yuan
Adam Trischler
Jie Lin
Vijay Chandrasekhar
Christopher Pal
Hao Dong
2020-04-18
ArXiv (prépublication)
openreview.net
openreview.net
Revision in Continuous Space: Unsupervised Text Style Transfer without Adversarial Learning
Dayiheng Liu
Jie Fu
Yidan Zhang
Christopher Pal
Jiancheng Lv
Typical methods for unsupervised text style transfer often rely on two key ingredients: 1) seeking the explicit disentanglement of the conte… (voir plus)nt and the attributes, and 2) troublesome adversarial learning. In this paper, we show that neither of these components is indispensable. We propose a new framework that utilizes the gradients to revise the sentence in a continuous space during inference to achieve text style transfer. Our method consists of three key components: a variational auto-encoder (VAE), some attribute predictors (one for each attribute), and a content predictor. The VAE and the two types of predictors enable us to perform gradient-based optimization in the continuous space, which is mapped from sentences in a discrete space, to find the representation of a target sentence with the desired attributes and preserved content. Moreover, the proposed method naturally has the ability to simultaneously manipulate multiple fine-grained attributes, such as sentence length and the presence of specific words, when performing text style transfer tasks. Compared with previous adversarial learning based methods, the proposed method is more interpretable, controllable and easier to train. Extensive experimental studies on three popular text style transfer tasks show that the proposed method significantly outperforms five state-of-the-art methods.
2020-04-02
Proceedings of the AAAI Conference on Artificial Intelligence (publié)
doi.org
arxiv.org
Interactive Language Learning by Question Answering
Xingdi Yuan
Marc-Alexandre Côté
Jie Fu
Zhouhan Lin
Christopher Pal
Yoshua Bengio
Adam Trischler
Humans observe and interact with the world to acquire knowledge. However, most existing machine reading comprehension (MRC) tasks miss the i… (voir plus)nteractive, information-seeking component of comprehension. Such tasks present models with static documents that contain all necessary information, usually concentrated in a single short substring. Thus, models can achieve strong performance through simple word- and phrase-based pattern matching. We address this problem by formulating a novel text-based question answering task: Question Answering with Interactive Text (QAit). In QAit, an agent must interact with a partially observable text-based environment to gather information required to answer questions. QAit poses questions about the existence, location, and attributes of objects found in the environment. The data is built using a text-based game generator that defines the underlying dynamics of interaction with the environment. We propose and evaluate a set of baseline models for the QAit task that includes deep reinforcement learning agents. Experiments show that the task presents a major challenge for machine reading systems, while humans solve it with relative ease.
2019-10-31
Conference on Empirical Methods in Natural Language Processing and International Joint Conference on Natural Language Processing (publié)
doi.org
arxiv.org
Structure Learning for Neural Module Networks
Vardaan Pahuja
Jie Fu
A. Chandar
Christopher Pal
Neural Module Networks, originally proposed for the task of visual question answering, are a class of neural network architectures that invo… (voir plus)lve human-specified neural modules, each designed for a specific form of reasoning. In current formulations of such networks only the parameters of the neural modules and/or the order of their execution is learned. In this work, we further expand this approach and also learn the underlying internal structure of modules in terms of the ordering and combination of simple and elementary arithmetic operators. We utilize a minimum amount of prior knowledge from the human-specified neural modules in the form of different input types and arithmetic operators used in these modules. Our results show that one is indeed able to simultaneously learn both internal module structure and module sequencing without extra supervisory signals for module execution sequencing. With this approach, we report performance comparable to models using hand-designed modules. In addition, we do a analysis of sensitivity of the learned modules w.r.t. the arithmetic operations and infer the analytical expressions of the learned modules.
2019-10-31
Proceedings of the Beyond Vision and LANguage: inTEgrating Real-world kNowledge (LANTERN) (publié)
doi.org
arxiv.org
Learning Sparse Mixture of Experts for Visual Question Answering
Vardaan Pahuja
Jie Fu
Christopher Pal
2019-09-18
ArXiv (prépublication)
arxiv.org
arxiv.org
Learning Multi-Task Communication with Message Passing for Sequence Learning
Pengfei Liu
Jie Fu
Yue Dong
Xipeng Qiu
Jackie CK Cheung
We present two architectures for multi-task learning with neural sequence models. Our approach allows the relationships between different ta… (voir plus)sks to be learned dynamically, rather than using an ad-hoc pre-defined structure as in previous work. We adopt the idea from message-passing graph neural networks, and propose a general graph multi-task learning framework in which different tasks can communicate with each other in an effective and interpretable way. We conduct extensive experiments in text classification and sequence labelling to evaluate our approach on multi-task learning and transfer learning. The empirical results show that our models not only outperform competitive baselines, but also learn interpretable and transferable patterns across tasks.
2019-07-16
Proceedings of the AAAI Conference on Artificial Intelligence (publié)
doi.org
Conditional Computation for Continual Learning
Min Lin
Jie Fu
Yoshua Bengio
Catastrophic forgetting of connectionist neural networks is caused by the global sharing of parameters among all training examples. In this … (voir plus)study, we analyze parameter sharing under the conditional computation framework where the parameters of a neural network are conditioned on each input example. At one extreme, if each input example uses a disjoint set of parameters, there is no sharing of parameters thus no catastrophic forgetting. At the other extreme, if the parameters are the same for every example, it reduces to the conventional neural network. We then introduce a clipped version of maxout networks which lies in the middle, i.e. parameters are shared partially among examples. Based on the parameter sharing analysis, we can locate a limited set of examples that are interfered when learning a new example. We propose to perform rehearsal on this set to prevent forgetting, which is termed as conditional rehearsal. Finally, we demonstrate the effectiveness of the proposed method in an online non-stationary setup, where updates are made after each new example and the distribution of the received example shifts over time.
2019-06-15
arXiv (prépublication)
doi.org
arxiv.org