Jian Tang

Biography

Jian Tang is an Associate professor at HEC's Department of Decision Sciences. He is also an Adjunct professor at the Department of Computer Science and Operations Research at University of Montreal and a Core Academic member at Mila - Quebec AI Institute. He is a Canada CIFAR AI Chair and the Founder of BioGeometry, an AI startup that focuses on generative AI for antibody discovery. Tang’s main research interests are deep generative models and graph machine learning, and their applications to drug discovery. He is an international leader in graph machine learning, and LINE, his node representation method, has been widely recognized and cited more than five thousand times. He has also done pioneering work on AI for drug discovery, such as developing the first open-source machine learning frameworks for drug discovery, TorchDrug and TorchProtein.

Current Students

Huiyu Cai

PhD - Université de Montréal

huiyu.cai@mila.quebec

Zewen Chi

Research Intern - Beijing Institute of Technology

Andreea-Ioana Deac

PhD - Université de Montréal

Research Intern - HEC Montréal

Michael Galkin

Collaborating researcher

Farzaneh Heidari

PhD - Université de Montréal

Principal supervisor :

Guillaume Rabusseau

Jerry Lu

PhD - Université de Montréal

Meng Qu

PhD - Université de Montréal

Chence Shi

PhD - Université de Montréal

Chuanru Wang

Master's Research - Université de Montréal

Sophie Xhonneux

PhD - Université de Montréal

Principal supervisor :

lpxhonneux@gmail.com

Minghao Xu

Collaborating researcher

Xinyu Yuan

PhD - Université de Montréal

Zhihao Zhan

PhD - Université de Montréal

Zuobai Zhang

PhD - Université de Montréal

Jianan Zhao

PhD - Université de Montréal

PhD - Université de Montréal

Publications

The Heterophilic Graph Learning Handbook: Benchmarks, Models, Theoretical Analysis, Applications and Challenges

Sitao Luan

Chenqing Hua

Qincheng Lu

Liheng Ma

Lirong Wu

Xinyu Wang

Minkai Xu

Xiao-Wen Chang

Doina Precup

Rex Ying

Stan Z. Li

Guy Wolf

Stefanie Jegelka

Homophily principle, \ie{} nodes with the same labels or similar attributes are more likely to be connected, has been commonly believed to b… (see more)e the main reason for the superiority of Graph Neural Networks (GNNs) over traditional Neural Networks (NNs) on graph-structured data, especially on node-level tasks. However, recent work has identified a non-trivial set of datasets where GNN's performance compared to the NN's is not satisfactory. Heterophily, i.e. low homophily, has been considered the main cause of this empirical observation. People have begun to revisit and re-evaluate most existing graph models, including graph transformer and its variants, in the heterophily scenario across various kinds of graphs, e.g. heterogeneous graphs, temporal graphs and hypergraphs. Moreover, numerous graph-related applications are found to be closely related to the heterophily problem. In the past few years, considerable effort has been devoted to studying and addressing the heterophily issue. In this survey, we provide a comprehensive review of the latest progress on heterophilic graph learning, including an extensive summary of benchmark datasets and evaluation of homophily metrics on synthetic graphs, meticulous classification of the most updated supervised and unsupervised learning methods, thorough digestion of the theoretical analysis on homophily/heterophily, and broad exploration of the heterophily-related applications. Notably, through detailed experiments, we are the first to categorize benchmark heterophilic datasets into three sub-categories: malignant, benign and ambiguous heterophily. Malignant and ambiguous datasets are identified as the real challenging datasets to test the effectiveness of new models on the heterophily challenge. Finally, we propose several challenges and future directions for heterophilic graph representation learning.

2024-07-12

ArXiv (preprint)

In-Context Learning, Can It Break Safety?

Sophie Xhonneux

David Dobre

Michael Noukhovitch

Dhanya Sridhar

2024-06-28

ICML.cc/2024/Workshop/NextGenAISafety (poster)

Augmenting Evolutionary Models with Structure-based Retrieval

Yining Huang

Zuobai Zhang

Debora Susan Marks

Pascal Notin

2024-06-17

ICML.cc/2024/Workshop/ML4LMS (poster)

GraphAny: A Foundation Model for Node Classification on Any Graph

Jianan Zhao

Hesham Mostafa

Mikhail Galkin

Michael Bronstein

Zhaocheng Zhu

2024-05-30

ArXiv (preprint)

The 1st International Workshop on Graph Foundation Models (GFM)

Haitao Mao

Jianan Zhao

Xiaoxin He

Zhikai Chen

Qian Huang

Zhaocheng Zhu

Micheal Bronstein

Xavier Bresson

Bryan Hooi

Haiyang Zhang

Xianfeng Tang

Luo Chen

Jiliang Tang

2024-05-13

Companion Proceedings of the ACM on Web Conference 2024 (published)

Zero-shot Logical Query Reasoning on any Knowledge Graph

Mikhail Galkin

Jincheng Zhou

Bruno Ribeiro

Zhaocheng Zhu

Complex logical query answering (CLQA) in knowledge graphs (KGs) goes beyond simple KG completion and aims at answering compositional querie… (see more)s comprised of multiple projections and logical operations. Existing CLQA methods that learn parameters bound to certain entity or relation vocabularies can only be applied to the graph they are trained on which requires substantial training time before being deployed on a new graph. Here we present UltraQuery, an inductive reasoning model that can zero-shot answer logical queries on any KG. The core idea of UltraQuery is to derive both projections and logical operations as vocabulary-independent functions which generalize to new entities and relations in any KG. With the projection operation initialized from a pre-trained inductive KG reasoning model, UltraQuery can solve CLQA on any KG even if it is only finetuned on a single dataset. Experimenting on 23 datasets, UltraQuery in the zero-shot inference mode shows competitive or better query answering performance than best available baselines and sets a new state of the art on 14 of them.

2024-04-10

ArXiv (preprint)

Prioritizing Safeguarding Over Autonomy: Risks of LLM Agents for Science

Xiangru Tang

Qiao Jin

Kunlun Zhu

Tongxin Yuan

Yichi Zhang

Wangchunshu Zhou

Meng Qu

Yilun Zhao

Zhuosheng Zhang

Arman Cohan

Zhiyong Lu

Mark Gerstein

2024-03-11

ICLR.cc/2024/Workshop/LLMAgents (poster)

Heterogeneous ensemble prediction model of CO emission concentration in municipal solid waste incineration process using virtual data and real data hybrid-driven

Runyu Zhang

Heng Xia

Jiakun Chen

Wen Yu

JunFei Qiao

2024-03-01

Journal of Cleaner Production (published)

Iterative Graph Self-Distillation

Hanlin Zhang

Shuai Lin

Weiyang Liu

Pan Zhou

Xiaodan Liang

Eric P. Xing

Recently, there has been increasing interest in the challenge of how to discriminatively vectorize graphs. To address this, we propose a met… (see more)hod called Iterative Graph Self-Distillation (IGSD) which learns graph-level representation in an unsupervised manner through instance discrimination using a self-supervised contrastive learning approach. IGSD involves a teacher-student distillation process that uses graph diffusion augmentations and constructs the teacher model using an exponential moving average of the student model. The intuition behind IGSD is to predict the teacher network representation of the graph pairs under different augmented views. As a natural extension, we also apply IGSD to semi-supervised scenarios by jointly regularizing the network with both supervised and self-supervised contrastive loss. Finally, we show that fine-tuning the IGSD-trained models with self-training can further improve graph representation learning. Empirically, we achieve significant and consistent performance gain on various graph datasets in both unsupervised and semi-supervised settings, which well validates the superiority of IGSD.

2024-03-01

IEEE Transactions on Knowledge and Data Engineering (published)

Deep Equilibrium Models For Algorithmic Reasoning

Sophie Xhonneux

Yu He

Andreea Deac

In this blogpost we discuss the idea of teaching neural networks to reach fixed points when reasoning. Specifically, on the algorithmic reas… (see more)oning benchmark CLRS the current neural networks are told the number of reasoning steps they need. While a quick fix is to add a termination network that predicts when to stop, a much more salient inductive bias is that the neural network shouldn't change it's answer any further once the answer is correct, i.e. it should reach a fixed point. This is supported by denotational semantics, which tells us that while loops that terminate are the minimum fixed points of a function. We implement this idea with the help of deep equilibrium models and discuss several hurdles one encounters along the way. We show on several algorithms from the CLRS benchmark the partial success of this approach and the difficulty in making it work robustly across all algorithms.

2024-02-16

ICLR.cc/2024/BlogPosts (published)

Deep Equilibrium Models For Algorithmic Reasoning

Sophie Xhonneux

Yu He

Andreea Deac

2024-02-16

ICLR.cc/2024/BlogPosts (accepted)

In-Context Learning Can Re-learn Forbidden Tasks

Sophie Xhonneux

David Dobre

Dhanya Sridhar

Despite significant investment into safety training, large language models (LLMs) deployed in the real world still suffer from numerous vuln… (see more)erabilities. One perspective on LLM safety training is that it algorithmically forbids the model from answering toxic or harmful queries. To assess the effectiveness of safety training, in this work, we study forbidden tasks, i.e., tasks the model is designed to refuse to answer. Specifically, we investigate whether in-context learning (ICL) can be used to re-learn forbidden tasks despite the explicit fine-tuning of the model to refuse them. We first examine a toy example of refusing sentiment classification to demonstrate the problem. Then, we use ICL on a model fine-tuned to refuse to summarise made-up news articles. Finally, we investigate whether ICL can undo safety training, which could represent a major security risk. For the safety task, we look at Vicuna-7B, Starling-7B, and Llama2-7B. We show that the attack works out-of-the-box on Starling-7B and Vicuna-7B but fails on Llama2-7B. Finally, we propose an ICL attack that uses the chat template tokens like a prompt injection attack to achieve a better attack success rate on Vicuna-7B and Starling-7B. Trigger Warning: the appendix contains LLM-generated text with violence, suicide, and misinformation.

2024-02-08

ArXiv (preprint)