Jian Tang

farzaneh.heidari@mila.quebec

Biography

Jian Tang is an Associate professor at HEC's Department of Decision Sciences. He is also an Adjunct professor at the Department of Computer Science and Operations Research at University of Montreal and a Core Academic member at Mila - Quebec AI Institute. He is a Canada CIFAR AI Chair and the Founder of BioGeometry, an AI startup that focuses on generative AI for antibody discovery. Tang’s main research interests are deep generative models and graph machine learning, and their applications to drug discovery. He is an international leader in graph machine learning, and LINE, his node representation method, has been widely recognized and cited more than five thousand times. He has also done pioneering work on AI for drug discovery, such as developing the first open-source machine learning frameworks for drug discovery, TorchDrug and TorchProtein.

Current Students

Huiyu Cai

PhD - Université de Montréal

huiyu.cai@mila.quebec

PhD - Université de Montréal

Principal supervisor :

Guillaume Rabusseau

Xixian Liu

PhD - Université de Montréal

xixian.liu@mila.quebec

Jiarui Lu

PhD - Université de Montréal

jiarui.lu@mila.quebec

PhD - Université de Montréal

Principal supervisor :

Gauthier Gidel

lpxhonneux@gmail.com

Jinxi Yang

Collaborating researcher - Wuhan University

jinxi.yang@mila.quebec

Xinyu Yuan

PhD - Université de Montréal

xinyu.yuan@mila.quebec

Zhihao Zhan

PhD - Université de Montréal

zhihao.zhan@mila.quebec

Zuobai Zhang

PhD - Université de Montréal

zuobai.zhang@mila.quebec

PhD - HEC Montréal

ya-shi.zhang@mila.quebec

Jianan Zhao

PhD - Université de Montréal

jianan.zhao@mila.quebec

Publications

Fusing Neural and Physical: Augment Protein Conformation Sampling with Tractable Simulations

Bozitao Zhong

The protein dynamics are common and important for their biological functions and properties, the study of which usually involves time-consum… (see more)ing molecular dynamics (MD) simulations *in silico*. Recently, generative models has been leveraged as a surrogate sampler to obtain conformation ensembles with orders of magnitude faster and without requiring any simulation data (a "zero-shot" inference). However, being agnostic of the underlying energy landscape, the accuracy of such generative model may still be limited. In this work, we explore the few-shot setting of such pre-trained generative sampler which incorporates MD simulations in a tractable manner. Specifically, given a target protein of interest, we first acquire some seeding conformations from the pre-trained sampler followed by a number of physical simulations in parallel starting from these seeding samples. Then we fine-tuned the generative model using the simulation trajectories above to become a target-specific sampler. Experimental results demonstrated the superior performance of such few-shot conformation sampler at a tractable computational cost.

2024-03-03

ICLR.cc/2024/Workshop/GEM (poster)

Structure-Informed Protein Language Model

Zuobai Zhang

Jiarui Lu

Vijil Chenthamarakshan

Aurelie Lozano

Payel Das

Protein language models are a powerful tool for learning protein representations through pre-training on vast protein sequence datasets. Ho… (see more)wever, traditional protein language models lack explicit structural supervision, despite its relevance to protein function. To address this issue, we introduce the integration of remote homology detection to distill structural information into protein language models without requiring explicit protein structures as input. We evaluate the impact of this structure-informed training on downstream protein function prediction tasks. Experimental results reveal consistent improvements in function annotation accuracy for EC number and GO term prediction. Performance on mutant datasets, however, varies based on the relationship between targeted properties and protein structures. This underscores the importance of considering this relationship when applying structure-aware training to protein function prediction tasks. Code and model weights will be made available upon acceptance.

2024-03-03

GEM @ International Conference on Learning Representations (poster)

Iterative Graph Self-Distillation

Hanlin Zhang

Shuai Lin

Weiyang Liu

Pan Zhou

Xiaodan Liang

Eric P. Xing

Recently, there has been increasing interest in the challenge of how to discriminatively vectorize graphs. To address this, we propose a met… (see more)hod called Iterative Graph Self-Distillation (IGSD) which learns graph-level representation in an unsupervised manner through instance discrimination using a self-supervised contrastive learning approach. IGSD involves a teacher-student distillation process that uses graph diffusion augmentations and constructs the teacher model using an exponential moving average of the student model. The intuition behind IGSD is to predict the teacher network representation of the graph pairs under different augmented views. As a natural extension, we also apply IGSD to semi-supervised scenarios by jointly regularizing the network with both supervised and self-supervised contrastive loss. Finally, we show that fine-tuning the IGSD-trained models with self-training can further improve graph representation learning. Empirically, we achieve significant and consistent performance gain on various graph datasets in both unsupervised and semi-supervised settings, which well validates the superiority of IGSD.

2024-02-29

IEEE Transactions on Knowledge and Data Engineering (published)

NOx emissions prediction for MSWI process based on dynamic modular neural network

Haoshan Duan

Xi Meng

JunFei Qiao

2024-02-29

Expert Systems with Applications (published)

Deep Equilibrium Models For Algorithmic Reasoning

Andreea Deac

In this blogpost we discuss the idea of teaching neural networks to reach fixed points when reasoning. Specifically, on the algorithmic reas… (see more)oning benchmark CLRS the current neural networks are told the number of reasoning steps they need. While a quick fix is to add a termination network that predicts when to stop, a much more salient inductive bias is that the neural network shouldn't change it's answer any further once the answer is correct, i.e. it should reach a fixed point. This is supported by denotational semantics, which tells us that while loops that terminate are the minimum fixed points of a function. We implement this idea with the help of deep equilibrium models and discuss several hurdles one encounters along the way. We show on several algorithms from the CLRS benchmark the partial success of this approach and the difficulty in making it work robustly across all algorithms.

2024-02-15

ICLR.cc/2024/BlogPosts (accepted)

In-Context Learning Can Re-learn Forbidden Tasks

Despite significant investment into safety training, large language models (LLMs) deployed in the real world still suffer from numerous vuln… (see more)erabilities. One perspective on LLM safety training is that it algorithmically forbids the model from answering toxic or harmful queries. To assess the effectiveness of safety training, in this work, we study forbidden tasks, i.e., tasks the model is designed to refuse to answer. Specifically, we investigate whether in-context learning (ICL) can be used to re-learn forbidden tasks despite the explicit fine-tuning of the model to refuse them. We first examine a toy example of refusing sentiment classification to demonstrate the problem. Then, we use ICL on a model fine-tuned to refuse to summarise made-up news articles. Finally, we investigate whether ICL can undo safety training, which could represent a major security risk. For the safety task, we look at Vicuna-7B, Starling-7B, and Llama2-7B. We show that the attack works out-of-the-box on Starling-7B and Vicuna-7B but fails on Llama2-7B. Finally, we propose an ICL attack that uses the chat template tokens like a prompt injection attack to achieve a better attack success rate on Vicuna-7B and Starling-7B. Trigger Warning: the appendix contains LLM-generated text with violence, suicide, and misinformation.

2024-02-07

ArXiv (preprint)

arxiv.org

Design and Application of Adaptive Sparse Deep Echo State Network

Cuili Yang

Sheng Yang

Bing Li

The prediction of appliances energy consumption in building belongs to time series forecasting problem, which can be solved by echo state ne… (see more)twork (ESN). However, due to the randomly initialized inputs and reservoir, some redundant or irrelevant components are inevitably generated in original ESN. To solve this problem, the adaptive sparse deep echo state network (ASDESN) is proposed, in which the information is processed layer by layer. Firstly, the principal component analysis (PCA) layer is inserted to penalize the redundant projection transmitted between sub-reservoirs. Secondly, the coordinate descent based adaptive sparse learning method is proposed to generate the sparse output weights. Particularly, the designed adaptive threshold strategy is able to enlarge the sparsity of output weights as network depth increases. Moreover, the echo state property (ESP) of ASDESN is given to ensure its applications. The experiment results in both simulated benchmark and real appliances energy datasets illustrate that the proposed ASDESN outperforms other ESNs with higher prediction accuracy and stability.

2024-01-31

IEEE transactions on consumer electronics (published)

Heterogeneous ensemble prediction model of CO emission concentration in municipal solid waste incineration process using virtual data and real data hybrid-driven

Runyu Zhang

Heng Xia

Jiakun Chen

Wen Yu

JunFei Qiao

2024-01-31

Journal of Cleaner Production (published)

Machine Learning Informed Diagnosis for Congenital Heart Disease in Large Claims Data Source

Ariane J. Marelli

Chao Li

Aihua Liu

Hanh Nguyen

Harry Moroz

James M. Brophy

Liming Guo

David L. Buckeridge

Archer Y. Yang

Yue Li

2024-01-31

JACC: Advances (published)

Unsupervised Discovery of Steerable Factors When Graph Deep Generative Models Are Entangled

Shengchao Liu

Chengpeng Wang

Jiarui Lu

Weili Nie

Hanchen Wang

Zhuoxinran Li

Bolei Zhou

2024-01-29

TMLR (accepted)

Evaluating Representation Learning on the Protein Structure Universe

Arian R. Jamasb

Alex Morehead

Chaitanya K. Joshi

Zuobai Zhang

Kieran Didi

Simon Mathis

Charles Harris

Jianlin Cheng

Pietro Lio

Tom L. Blundell

We introduce ProteinWorkshop, a comprehensive benchmark suite for representation learning on protein structures with Geometric Graph Neural … (see more)Networks. We consider large-scale pre-training and downstream tasks on both experimental and predicted structures to enable the systematic evaluation of the quality of the learned structural representation and their usefulness in capturing functional relationships for downstream tasks. We find that: (1) large-scale pretraining on AlphaFold structures and auxiliary tasks consistently improve the performance of both rotation-invariant and equivariant GNNs, and (2) more expressive equivariant GNNs benefit from pretraining to a greater extent compared to invariant models. We aim to establish a common ground for the machine learning and computational biology communities to rigorously compare and advance protein structure representation learning. Our open-source codebase reduces the barrier to entry for working with large protein structure datasets by providing: (1) storage-efficient dataloaders for large-scale structural databases including AlphaFoldDB and ESM Atlas, as well as (2) utilities for constructing new tasks from the entire PDB. ProteinWorkshop is available at: github.com/a-r-j/ProteinWorkshop.

2024-01-15

ICLR.cc/2024/Conference (poster)

Towards Foundation Models for Knowledge Graph Reasoning

Hesham Mostafa

Foundation models in language and vision have the ability to run inference on any textual and visual inputs thanks to the transferable repre… (see more)sentations such as a vocabulary of tokens in language. Knowledge graphs (KGs) have different entity and relation vocabularies that generally do not overlap. The key challenge of designing foundation models on KGs is to learn such transferable representations that enable inference on any graph with arbitrary entity and relation vocabularies. In this work, we make a step towards such foundation models and present ULTRA, an approach for learning universal and transferable graph representations. ULTRA builds relational representations as a function conditioned on their interactions. Such a conditioning strategy allows a pre-trained ULTRA model to inductively generalize to any unseen KG with any relation vocabulary and to be fine-tuned on any graph. Conducting link prediction experiments on 57 different KGs, we find that the zero-shot inductive inference performance of a single pre-trained ULTRA model on unseen graphs of various sizes is often on par or better than strong baselines trained on specific graphs. Fine-tuning further boosts the performance.

2024-01-15

ICLR.cc/2024/Conference (poster)