Portrait of Linfeng Du

Linfeng Du

linfeng.du@mila.quebec
PhD - McGill University
Supervisor
Xue (Steve) Liu
Research Topics
Large Language Models (LLM)
Retrieval-Augmented LLMs
Time Series Forecasting
Google Scholar
GitHub

Publications

Discrete Diffusion Models: A Unified Framework from Tokenization to Generation
Ye Yuan
Weien Li
Rui Song
Zeyu Li
Haochen Liu
Xiangyu Kong
Zixuan Dong
Linfeng Du
Zipeng Sun
Weixu Zhang
Jiaxin Huang
Changjiang Han
Yonghan Yang
Zichen Zhao
Xiuyuan Hu
Haolun Wu
Yankai Chen
Fengran Mo
Jikun Kang
Bowei He … (see 2 more)
Philip S. Yu
Xue Liu
Discrete denoising diffusion models (DDMs) have recently emerged as a compelling alternative to autoregressive (AR) modeling for discrete da… (see more)ta, offering parallel generation and iterative global refinement capabilities. Unlike continuous diffusion, where the state space is fixed, DDMs are fundamentally shaped by how the discrete state space is constructed: the tokenization scheme, the vocabulary topology, and domain-specific structural alphabets. This work introduces a unified conceptual framework that views discrete diffusion models through the construction of the underlying discrete state space. Within this framework, existing formulations, including transition-matrix, masking/absorbing-state, and score/ratio-based approaches, emerge as different instantiations of a common design space. The framework further exposes common design trade-offs across training objectives, inference algorithms, scaling behavior, systems optimization, and evaluation protocols, suggesting several promising directions for future research.
2026-07-14
arXiv (preprint)
doi.org
arxiv.org
Preference Heads in Large Language Models: A Mechanistic Framework for Interpretable Personalization
Weixu Zhang
Ye Yuan
Changjiang Han
Yuxing Tian
Zipeng Sun
Linfeng Du
Jikun Kang
Hong Kang
Xue Liu
Haolun Wu
Large Language Models (LLMs) exhibit strong implicit personalization ability, yet most existing approaches treat this behavior as a black bo… (see more)x, relying on prompt engineering or fine tuning on user data. In this work, we adopt a mechanistic interpretability perspective and hypothesize the existence of a sparse set of Preference Heads, attention heads that encode user specific stylistic and topical preferences and exert a causal influence on generation. We introduce Differential Preference Steering (DPS), a training free framework that (1) identifies Preference Heads through causal masking analysis and (2) leverages them for controllable and interpretable personalization at inference time. DPS computes a Preference Contribution Score (PCS) for each attention head, directly measuring its causal impact on user aligned outputs. During decoding, we contrast model predictions with and without Preference Heads, amplifying the difference between personalized and generic logits to selectively strengthen preference aligned continuations. Experiments on widely used personalization benchmarks across multiple LLMs demonstrate consistent gains in personalization fidelity while preserving content coherence and low computational overhead. Beyond empirical improvements, DPS provides a mechanistic explanation of where and how personalization emerges within transformer architectures. Our implementation is publicly available.
2026-04-23
arXiv (preprint)
doi.org
arxiv.org
Optimizing User Profiles via Contextual Bandits for Retrieval-Augmented LLM Personalization
Linfeng Du
Ye Yuan
Zichen Zhao
Fuyuan Lyu
Emiliano Penaloza
Xiuying Chen
Zipeng Sun
Jikun Kang
Laurent Charlin
Xue Liu
Haolun Wu
Large Language Models (LLMs) excel at general-purpose tasks, yet adapting their responses to individual users remains challenging. Retrieval… (see more) augmentation provides a lightweight alternative to fine-tuning by conditioning LLMs on user history records, and existing approaches typically select these records based on semantic relevance. We argue that relevance serves as an unreliable proxy for utility: a record may be semantically similar to a query yet fail to improve generation quality or even degrade it due to redundancy or conflicting information. To bridge this gap, we propose PURPLE, a contextual bandit framework that oPtimizes UseR Profiles for Llm pErsonalization. In contrast to a greedy selection of the most relevant records, PURPLE treats profile construction as a set generation process and utilizes a Plackett-Luce ranking model to capture complex inter-record dependencies. By training with dense feedback provided by the likelihood of the reference response, our method aligns retrieval directly with generation quality. Extensive experiments on nine personalization tasks demonstrate that PURPLE consistently outperforms strong heuristic and retrieval-augmented baselines in both effectiveness and efficiency, establishing a principled and scalable solution for optimizing user profiles.
2025-12-31
arXiv (preprint)
doi.org
arxiv.org
Support-Proximity Augmented Diffusion Estimation for Offline Black-Box Optimization
Yonghan Yang
Ye Yuan
Zipeng Sun
Linfeng Du
Bowei He
Haolun Wu
Can Chen
Xue Liu
Offline black-box optimization aims to discover novel designs with high property scores using only a static dataset, a task fundamentally ch… (see more)allenged by the out-of-distribution (OOD) extrapolation problem. Existing approaches typically bifurcate into inverse methods, which struggle with the ill-posed nature of mapping scores to designs, and forward methods, which often lack the distributional expressivity to quantify uncertainty effectively. In this work, we propose \textbf{SPADE} (\textbf{S}upport-\textbf{P}roximity \textbf{A}ugmented \textbf{D}iffusion \textbf{E}stimation), a novel framework that reimagines forward surrogate modeling through the lens of conditional generative modeling. SPADE models the forward likelihood
2025-12-31
International Conference on Machine Learning (Accept (regular))
doi.org
openreview.net