Portrait de Yinhan Lu

Yinhan Lu

yinhan.lu@mila.quebec
Collaborateur·rice de recherche - McGill
Superviseur⋅e principal⋅e
David Ifeoluwa Adelani
Sujets de recherche
Traitement du langage naturel
Site web
Google Scholar
GitHub

Publications

Simplifying the Modeling of Arbitrary Conditionals in Natural Language
Yinhan Lu
Eric Elmoznino
Leo Gagnon
Sarthak Mittal
Tejas Kasetty
Guillaume Lajoie
Causal Transformers model sequences through an autoregressive factorization of the joint distribution, which enables efficient left-to-right… (voir plus) decoding and conditional likelihood computation. However, they cannot tractably sample from or evaluate arbitrary conditionals -- e.g., a block of text conditioned on past and future tokens. Recent work aims to solve this problem through novel architectures, but they often lead to sub-optimal modeling of such conditionals and degraded generations. We propose Arbitrary Conditionals GPT (AC-GPT) which introduces a simple modification to standard causal Transformers to enable evaluating and sampling from arbitrary conditionals -- including past, future, and mixed contexts -- within a single forward pass. Unlike prior approaches, our method preserves the standard left-to-right ordering and next-token prediction objective essential for both strong performance and efficient training on natural language. Crucially, this compatibility allows existing LLMs to be fine-tuned for arbitrary conditioning. Our empirical results indicate that our method outperforms baselines on modeling arbitrary conditionals, without degrading standard left-to-right performance.
2026-06-11
arXiv (prépublication)
doi.org
arxiv.org