Jian Tang

Zewen Chi

Research Intern - Beijing Institute of Technology

zewen.chi@mila.quebec

Andreea-Ioana Deac

PhD - Université de Montréal

deacandr@mila.quebec

Michael Galkin

Collaborating Researcher

mikhail.galkin@mila.quebec

Farzaneh Heidari

PhD - Université de Montréal

Principal supervisor :

Guillaume Rabusseau

farzaneh.heidari@mila.quebec

Jerry Lu

PhD - Université de Montréal

jiarui.lu@mila.quebec

Meng Qu

PhD - Université de Montréal

qumeng@mila.quebec

Chence Shi

PhD - Université de Montréal

chence.shi@mila.quebec

Chuanru Wang

Master's Research - Université de Montréal

chuanrui.wang@mila.quebec

Sophie (Louis-Pascal) Xhonneux

PhD - Université de Montréal

Co-supervisor :

sophie.xhonneux@mila.quebec

Minghao Xu

Collaborating Researcher

minghao.xu@mila.quebec

Xinyu Yuan

PhD - Université de Montréal

xinyu.yuan@mila.quebec

Zhihao Zhan

PhD - Université de Montréal

zhihao.zhan@mila.quebec

Zuobai Zhang

PhD - Université de Montréal

zuobai.zhang@mila.quebec

Yangtian Zhang

Collaborating Researcher

yangtian.zhang@mila.quebec

Jianan Zhao

PhD - Université de Montréal

jianan.zhao@mila.quebec

Zhaocheng Zhu

PhD - Université de Montréal

zhuzhaoc@mila.quebec

Publications

Zero-shot Logical Query Reasoning on any Knowledge Graph

Mikhail Galkin

Jincheng Zhou

Bruno Ribeiro

Zhaocheng Zhu

Complex logical query answering (CLQA) in knowledge graphs (KGs) goes beyond simple KG completion and aims at answering compositional querie… (see more)s comprised of multiple projections and logical operations. Existing CLQA methods that learn parameters bound to certain entity or relation vocabularies can only be applied to the graph they are trained on which requires substantial training time before being deployed on a new graph. Here we present UltraQuery, an inductive reasoning model that can zero-shot answer logical queries on any KG. The core idea of UltraQuery is to derive both projections and logical operations as vocabulary-independent functions which generalize to new entities and relations in any KG. With the projection operation initialized from a pre-trained inductive KG reasoning model, UltraQuery can solve CLQA on any KG even if it is only finetuned on a single dataset. Experimenting on 23 datasets, UltraQuery in the zero-shot inference mode shows competitive or better query answering performance than best available baselines and sets a new state of the art on 14 of them.

2024-04-10

ArXiv (preprint)

arxiv.org

Prioritizing Safeguarding Over Autonomy: Risks of LLM Agents for Science

Xiangru Tang

Qiao Jin

Kunlun Zhu

Tongxin Yuan

Yichi Zhang

Wangchunshu Zhou

Meng Qu

Yilun Zhao

Zhuosheng Zhang

Arman Cohan

Zhiyong Lu

Mark Gerstein

2024-03-11

ICLR.cc/2024/Workshop/LLMAgents (poster)

Heterogeneous ensemble prediction model of CO emission concentration in municipal solid waste incineration process using virtual data and real data hybrid-driven

Runyu Zhang

Heng Xia

Jiakun Chen

Wen Yu

JunFei Qiao

2024-03-01

Journal of Cleaner Production (published)

Iterative Graph Self-Distillation

Hanlin Zhang

Shuai Lin

Weiyang Liu

Pan Zhou

Xiaodan Liang

Eric P. Xing

Recently, there has been increasing interest in the challenge of how to discriminatively vectorize graphs. To address this, we propose a met… (see more)hod called Iterative Graph Self-Distillation (IGSD) which learns graph-level representation in an unsupervised manner through instance discrimination using a self-supervised contrastive learning approach. IGSD involves a teacher-student distillation process that uses graph diffusion augmentations and constructs the teacher model using an exponential moving average of the student model. The intuition behind IGSD is to predict the teacher network representation of the graph pairs under different augmented views. As a natural extension, we also apply IGSD to semi-supervised scenarios by jointly regularizing the network with both supervised and self-supervised contrastive loss. Finally, we show that fine-tuning the IGSD-trained models with self-training can further improve graph representation learning. Empirically, we achieve significant and consistent performance gain on various graph datasets in both unsupervised and semi-supervised settings, which well validates the superiority of IGSD.

2024-03-01

IEEE Transactions on Knowledge and Data Engineering (published)

arxiv.org

Deep Equilibrium Models For Algorithmic Reasoning

Sophie Xhonneux

Yu He

Andreea Deac

In this blogpost we discuss the idea of teaching neural networks to reach fixed points when reasoning. Specifically, on the algorithmic reas… (see more)oning benchmark CLRS the current neural networks are told the number of reasoning steps they need. While a quick fix is to add a termination network that predicts when to stop, a much more salient inductive bias is that the neural network shouldn't change it's answer any further once the answer is correct, i.e. it should reach a fixed point. This is supported by denotational semantics, which tells us that while loops that terminate are the minimum fixed points of a function. We implement this idea with the help of deep equilibrium models and discuss several hurdles one encounters along the way. We show on several algorithms from the CLRS benchmark the partial success of this approach and the difficulty in making it work robustly across all algorithms.

2024-02-16

ICLR.cc/2024/BlogPosts (published)

Deep Equilibrium Models For Algorithmic Reasoning

Sophie Xhonneux

Yu He

Andreea Deac

2024-02-16

ICLR.cc/2024/BlogPosts (published)

In-Context Learning Can Re-learn Forbidden Tasks

Sophie Xhonneux

David Dobre

Dhanya Sridhar

Despite significant investment into safety training, large language models (LLMs) deployed in the real world still suffer from numerous vuln… (see more)erabilities. One perspective on LLM safety training is that it algorithmically forbids the model from answering toxic or harmful queries. To assess the effectiveness of safety training, in this work, we study forbidden tasks, i.e., tasks the model is designed to refuse to answer. Specifically, we investigate whether in-context learning (ICL) can be used to re-learn forbidden tasks despite the explicit fine-tuning of the model to refuse them. We first examine a toy example of refusing sentiment classification to demonstrate the problem. Then, we use ICL on a model fine-tuned to refuse to summarise made-up news articles. Finally, we investigate whether ICL can undo safety training, which could represent a major security risk. For the safety task, we look at Vicuna-7B, Starling-7B, and Llama2-7B. We show that the attack works out-of-the-box on Starling-7B and Vicuna-7B but fails on Llama2-7B. Finally, we propose an ICL attack that uses the chat template tokens like a prompt injection attack to achieve a better attack success rate on Vicuna-7B and Starling-7B. Trigger Warning: the appendix contains LLM-generated text with violence, suicide, and misinformation.

2024-02-08

ArXiv (preprint)

arxiv.org

Machine Learning Informed Diagnosis for Congenital Heart Disease in Large Claims Data Source

Ariane J. Marelli

Chao Li

Aihua Liu

Hanh Nguyen

Harry Moroz

James M. Brophy

Liming Guo

2024-02-01

JACC: Advances (published)

Unsupervised Discovery of Steerable Factors When Graph Deep Generative Models Are Entangled

Shengchao Liu

Chengpeng Wang

Jiarui Lu

Weili Nie

Hanchen Wang

Zhuoxinran Li

Bolei Zhou

2024-01-30

TMLR (accepted)

Evaluating Representation Learning on the Protein Structure Universe

Arian Rokkum Jamasb

Alex Morehead

Chaitanya K. Joshi

Zuobai Zhang

Kieran Didi

Simon V Mathis

Charles Harris

Jianlin Cheng

Pietro Lio

Tom Leon Blundell

2024-01-16

ICLR.cc/2024/Conference (poster)

Str2Str: A Score-based Framework for Zero-shot Protein Conformation Sampling

Jiarui Lu

Bozitao Zhong

Zuobai Zhang

2024-01-16

ICLR.cc/2024/Conference (poster)

Towards Foundation Models for Knowledge Graph Reasoning

Mikhail Galkin

Xinyu Yuan

Hesham Mostafa

Zhaocheng Zhu

Foundation models in language and vision have the ability to run inference on any textual and visual inputs thanks to the transferable repre… (see more)sentations such as a vocabulary of tokens in language. Knowledge graphs (KGs) have different entity and relation vocabularies that generally do not overlap. The key challenge of designing foundation models on KGs is to learn such transferable representations that enable inference on any graph with arbitrary entity and relation vocabularies. In this work, we make a step towards such foundation models and present ULTRA, an approach for learning universal and transferable graph representations. ULTRA builds relational representations as a function conditioned on their interactions. Such a conditioning strategy allows a pre-trained ULTRA model to inductively generalize to any unseen KG with any relation vocabulary and to be fine-tuned on any graph. Conducting link prediction experiments on 57 different KGs, we find that the zero-shot inductive inference performance of a single pre-trained ULTRA model on unseen graphs of various sizes is often on par or better than strong baselines trained on specific graphs. Fine-tuning further boosts the performance.

2024-01-16

ICLR.cc/2024/Conference (poster)