Portrait de Zhan Su n'est pas disponible

Zhan Su

Collaborateur·rice alumni - University of Copenhagen
Superviseur⋅e principal⋅e
Alessandro Sordoni
Sujets de recherche
Apprentissage profond

Publications

It Takes Two: Your GRPO Is Secretly DPO
Yihong Wu
Liheng Ma
Lei Ding
Muzhi Li
Xinyu Wang
Kejia Chen
Zhan Su
Zhanguang Zhang
Chenyang Huang
Yingxue Zhang
Mark J. Coates
Jian-Yun Nie
Group Relative Policy Optimization (GRPO) is a prominent reinforcement learning algorithm for post-training Large Language Models (LLMs). I… (voir plus)t is commonly believed that GRPO necessitates a large group size to ensure stable training via precise statistical estimation, which incurs substantial computational overhead. In this work, we challenge this assumption by reframing GRPO as a form of contrastive learning, which reveals a fundamental connection to Direct Preference Optimization (DPO). Motivated by DPO's empirical success, we investigate the minimal two-rollout case (2-GRPO)—a configuration previously deemed infeasible. We provide a rigorous theoretical analysis to validate 2-GRPO and demonstrate empirically that it achieves performance on par with 16-GRPO, despite using only
2025-10-15
NeurIPS.cc/2025/Workshop/ER (spotlight)
openreview.net
openreview.net
Exploring Sparse Adapters for Scalable Merging of Parameter Efficient Experts
Samin Yeasar Arnob
Zhan Su
Minseon Kim
Oleksiy Ostapenko
Riyasat Ohib
Esra'a Saleh
Doina Precup
Lucas Caccia
Alessandro Sordoni
Merging parameter-efficient task experts has recently gained growing attention as a way to build modular architectures that can be rapidly a… (voir plus)dapted on the fly for specific downstream tasks, without requiring additional fine-tuning. Typically, LoRA (Low-Rank Adaptation) serves as the foundational building block of such parameter-efficient modular architectures, leveraging low-rank weight structures to reduce the number of trainable parameters. In this paper, we study the properties of sparse adapters, which train only a subset of weights in the base neural network, as potential building blocks of modular architectures. First, we propose a simple method for training highly effective sparse adapters, which is conceptually simpler than existing methods in the literature and surprisingly outperforms both LoRA and full fine-tuning in our setting. Next, we investigate the merging properties of these sparse adapters by merging adapters for up to 20 natural language processing tasks, thus scaling beyond what is usually studied in the literature. Our findings demonstrate that sparse adapters yield superior in-distribution performance post-merging compared to LoRA or full model merging. Achieving strong held-out performance remains a challenge for all methods considered.
2025-07-06
colmweb.org/COLM/2025/Conference (accepté)
doi.org
openreview.net
Towards Modular LLMs by Building and Reusing a Library of LoRAs
Oleksiy Ostapenko
Zhan Su
Edoardo Maria Ponti
Laurent Charlin
Nicolas Le Roux
Matheus Pereira
Lucas Caccia
Alessandro Sordoni
The growing number of parameter-efficient adaptations of a base large language model (LLM) calls for studying whether we can reuse such trai… (voir plus)ned adapters to improve performance for new tasks. We study how to best build a library of adapters given multi-task data and devise techniques for both zero-shot and supervised task generalization through routing in such library. We benchmark existing approaches to build this library and introduce model-based clustering, MBC, a method that groups tasks based on the similarity of their adapter parameters, indirectly optimizing for transfer across the multi-task dataset. To re-use the library, we present a novel zero-shot routing mechanism, Arrow, which enables dynamic selection of the most relevant adapters for new inputs without the need for retraining. We experiment with several LLMs, such as Phi-2 and Mistral, on a wide array of held-out tasks, verifying that MBC-based adapters and Arrow routing lead to superior generalization to new tasks. We make steps towards creating modular, adaptable LLMs that can match or outperform traditional joint training.
2023-12-31
ICML (publié)
doi.org
proceedings.mlr.press
A Case Study of Instruction Tuning with Mixture of Parameter-Efficient Experts
O. Ostapenko
Lucas Caccia
Zhan Su
Nicolas Le Roux
Laurent Charlin
Alessandro Sordoni
2023-10-27
NeurIPS.cc/2023/Workshop/Instruction (publié)
openreview.net
openreview.net
Multi-Head Adapter Routing for Cross-Task Generalization
Lucas Caccia
Edoardo Ponti
Zhan Su
Matheus Pereira
Nicolas Le Roux
Alessandro Sordoni
Parameter-efficient fine-tuning (PEFT) for cross-task generalization consists in pre-training adapters on a multi-task training set before f… (voir plus)ew-shot adaptation to test tasks. Polytropon [Ponti et al., 2023] (
2023-09-20
NeurIPS.cc/2023/Conference (poster)
doi.org
openreview.net