Jian Tang

GraphCG: Unsupervised Discovery of Steerable Factors in Graphs

Shengchao Liu

Chengpeng Wang

Weili Nie

Hanchen Wang

Jiarui Lu

Bolei Zhou

Deep generative models have been extensively explored recently, especially for the graph data such as molecular graphs and point clouds. Yet… (voir plus), much less investigation has been carried out on understanding the learned latent space of deep graph generative models. Such understandings can open up a unified perspective and provide guidelines for essential tasks like controllable generation. In this paper, we first examine the representation space of the recent deep generative model trained for graph data, observing that the learned representation space is not perfectly disentangled. Based on this observation, we then propose an unsupervised method called GraphCG, which is model-agnostic and task-agnostic for discovering steerable factors in graph data. Specifically, GraphCG learns the semantic-rich directions via maximizing the corresponding mutual information, where the edited graph along the same direction will possess certain steerable factors. We conduct experiments on two types of graph data, molecular graphs and point clouds. Both the quantitative and qualitative results show the effectiveness of GraphCG for discovering steerable factors. The code will be public in the near future.

2022-11-22

NeurIPS.cc/2022/Workshop/GLFrontiers (publié)

openreview.net

Flaky Performances when Pretraining on Relational Databases

Shengchao Liu

David Vazquez

Pierre-Andre Noel

2022-11-09

ArXiv (prépublication)

Implications of Topological Imbalance for Representation Learning on Biomedical Knowledge Graphs

Stephen Bonner

Ufuk Kirik

Ola Engkvist

Ian P Barrett

Adoption of recently developed methods from machine learning has given rise to creation of drug-discovery knowledge graphs (KGs) that utiliz… (voir plus)e the interconnected nature of the domain. Graph-based modelling of the data, combined with KG embedding (KGE) methods, are promising as they provide a more intuitive representation and are suitable for inference tasks such as predicting missing links. One common application is to produce ranked lists of genes for a given disease, where the rank is based on the perceived likelihood of association between the gene and the disease. It is thus critical that these predictions are not only pertinent but also biologically meaningful. However, KGs can be biased either directly due to the underlying data sources that are integrated or due to modelling choices in the construction of the graph, one consequence of which is that certain entities can get topologically overrepresented. We demonstrate the effect of these inherent structural imbalances, resulting in densely connected entities being highly ranked no matter the context. We provide support for this observation across different datasets, models as well as predictive tasks. Further, we present various graph perturbation experiments which yield more support to the observation that KGE models can be more influenced by the frequency of entities rather than any biological information encoded within the relations. Our results highlight the importance of data modelling choices, and emphasizes the need for practitioners to be mindful of these issues when interpreting model outputs and during KG composition.

2022-07-26

Briefings in Bioinformatics (publié)

Subgraph Retrieval Enhanced Model for Multi-hop Knowledge Base Question Answering

Jing Zhang

Xiaokang Zhang

Jifan Yu

Jie Tang

Cuiping Li

Hong Chen

2022-05-01

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (publié)

Subgraph Retrieval Enhanced Model for Multi-hop Knowledge Base Question Answering

Jing Zhang

Xiaokang Zhang

Jifan Yu

Jie Tang

Cuiping Li

Hong Chen

Recent works on knowledge base question answering (KBQA) retrieve subgraphs for easier reasoning. The desired subgraph is crucial as a small… (voir plus) one may exclude the answer but a large one might introduce more noises. However, the existing retrieval is either heuristic or interwoven with the reasoning, causing reasoning on the partial subgraphs, which increases the reasoning bias when the intermediate supervision is missing. This paper proposes a trainable subgraph retriever (SR) decoupled from the subsequent reasoning process, which enables a plug-and-play framework to enhance any subgraph-oriented KBQA model. Extensive experiments demonstrate SR achieves significantly better retrieval and QA performance than existing retrieval methods. Via weakly supervised pre-training as well as the end-to-end fine-tuning, SR achieves new state-of-the-art performance when combined with NSM (He et al., 2021), a subgraph-oriented reasoner, for embedding-based KBQA methods. Codes and datasets are available online (https://github.com/RUCKBReasoning/SubgraphRetrievalKBQA)

2022-02-27

ArXiv (preprint)

High-Order Pooling for Graph Neural Networks with Tensor Decomposition

Chenqing Hua

Guillaume Rabusseau

openreview.net

Metro: Memory-Enhanced Transformer for Retrosynthetic Planning via Reaction Tree

Songtao Liu

Zhitao Ying

Rex Ying

Zuobai Zhang

Peilin Zhao

Lu Lin

Dinghao Wu

Retrosynthetic planning plays a critical role in drug discovery and organic chemistry. Starting from a target molecule as the root node, it … (voir plus)aims to find a complete reaction tree subject to the constraint that all leaf nodes belong to a set of starting materials. The multi-step reactions are crucial because they determine the flow chart in the production of the Organic Chemical Industry. However, existing datasets lack curation of tree-structured multi-step reactions, and fail to provide such reaction trees, limiting models’ understanding of organic molecule transformations. In this work, we first develop a benchmark curated for the retrosynthetic planning task, which consists of 124,869 reaction trees retrieved from the public USPTO-full dataset. On top of that, we propose Metro: Memory-Enhanced Transformer for RetrOsynthetic planning. Specifically, the dependency among molecules in the reaction tree is captured as context information for multi-step retrosynthesis predictions through transformers with a memory module. Extensive experiments show that Metro dramatically outperforms existing single-step retrosynthesis models by at least 10.7% in top-1 accuracy. The experiments demonstrate the superiority of exploiting context information in the retrosynthetic planning task. Moreover, the proposed model can be directly used for synthetic accessibility analysis, as it is trained on reaction trees with the shortest depths. Our work is the first step towards a brand new formulation for retrosynthetic planning in the aspects of data construction, model design, and evaluation. Code is available at https://github.com/SongtaoLiu0823/metro.

2022-01-01

arXiv.org (prépublication)

openreview.net

Implications of Topological Imbalance for Representation Learning on Biomedical Knowledge Graphs

Stephen Bonner

Ufuk Kirik

Ola Engkvist

I. Barrett

Adoption of recently developed methods from machine learning has given rise to creation of drug-discovery knowledge graphs (KGs) that utiliz… (voir plus)e the interconnected nature of the domain. Graph-based modelling of the data, combined with KG embedding (KGE) methods, are promising as they provide a more intuitive representation and are suitable for inference tasks such as predicting missing links. One common application is to produce ranked lists of genes for a given disease, where the rank is based on the perceived likelihood of association between the gene and the disease. It is thus critical that these predictions are not only pertinent but also biologically meaningful. However, KGs can be biased either directly due to the underlying data sources that are integrated or due to modelling choices in the construction of the graph, one consequence of which is that certain entities can get topologically overrepresented. We demonstrate the effect of these inherent structural imbalances, resulting in densely connected entities being highly ranked no matter the context. We provide support for this observation across different datasets, models as well as predictive tasks. Further, we present various graph perturbation experiments which yield more support to the observation that KGE models can be more influenced by the frequency of entities rather than any biological information encoded within the relations. Our results highlight the importance of data modelling choices, and emphasizes the need for practitioners to be mindful of these issues when interpreting model outputs and during KG composition.

2021-12-13

ArXiv (preprint)

Full-Scale Information Diffusion Prediction With Reinforced Recurrent Networks

Cheng Yang

Hao Wang

Chuan Shi

Maosong Sun

Ganqu Cui

Zhiyuan Liu

Information diffusion prediction is an important task, which studies how information items spread among users. With the success of deep lear… (voir plus)ning techniques, recurrent neural networks (RNNs) have shown their powerful capability in modeling information diffusion as sequential data. However, previous works focused on either microscopic diffusion prediction, which aims at guessing who will be the next influenced user at what time, or macroscopic diffusion prediction, which estimates the total numbers of influenced users during the diffusion process. To the best of our knowledge, few attempts have been made to suggest a unified model for both microscopic and macroscopic scales. In this article, we propose a novel full-scale diffusion prediction model based on reinforcement learning (RL). RL incorporates the macroscopic diffusion size information into the RNN-based microscopic diffusion model by addressing the nondifferentiable problem. We also employ an effective structural context extraction strategy to utilize the underlying social graph information. Experimental results show that our proposed model outperforms state-of-the-art baseline models on both microscopic and macroscopic diffusion predictions on three real-world datasets.

2021-09-01

IEEE Transactions on Neural Networks and Learning Systems (publié)

Full-Scale Information Diffusion Prediction With Reinforced Recurrent Networks

Cheng Yang

Hao Wang

Chuan Shi

Maosong Sun

Ganqu Cui

Zhiyuan Liu

Information diffusion prediction is an important task, which studies how information items spread among users. With the success of deep lear… (voir plus)ning techniques, recurrent neural networks (RNNs) have shown their powerful capability in modeling information diffusion as sequential data. However, previous works focused on either microscopic diffusion prediction, which aims at guessing who will be the next influenced user at what time, or macroscopic diffusion prediction, which estimates the total numbers of influenced users during the diffusion process. To the best of our knowledge, few attempts have been made to suggest a unified model for both microscopic and macroscopic scales. In this article, we propose a novel full-scale diffusion prediction model based on reinforcement learning (RL). RL incorporates the macroscopic diffusion size information into the RNN-based microscopic diffusion model by addressing the nondifferentiable problem. We also employ an effective structural context extraction strategy to utilize the underlying social graph information. Experimental results show that our proposed model outperforms state-of-the-art baseline models on both microscopic and macroscopic diffusion predictions on three real-world datasets.

2021-09-01

IEEE Transactions on Neural Networks and Learning Systems (published)

An End-to-End Framework for Molecular Conformation Generation via Bilevel Programming

Minkai Xu

Wujie Wang

Shitong Luo

Chence Shi

Yoshua Bengio

Rafael G'omez-bombarelli

Predicting molecular conformations (or 3D structures) from molecular graphs is a fundamental problem in many applications. Most existing app… (voir plus)roaches are usually divided into two steps by first predicting the distances between atoms and then generating a 3D structure through optimizing a distance geometry problem. However, the distances predicted with such two-stage approaches may not be able to consistently preserve the geometry of local atomic neighborhoods, making the generated structures unsatisfying. In this paper, we propose an end-to-end solution for molecular conformation prediction called ConfVAE based on the conditional variational autoencoder framework. Specifically, the molecular graph is first encoded in a latent space, and then the 3D structures are generated by solving a principled bilevel optimization program. Extensive experiments on several benchmark data sets prove the effectiveness of our proposed approach over existing state-of-the-art approaches. Code is available at https://github.com/MinkaiXu/ConfVAE-ICML21.

2021-07-01

Proceedings of the 38th International Conference on Machine Learning (publié)

proceedings.mlr.press