Portrait of Yinhan Lu

Yinhan Lu

yinhan.lu@mila.quebec
Collaborating researcher - McGill University
Supervisor
David Ifeoluwa Adelani
Research Topics
Natural Language Processing
Website
Google Scholar
GitHub

Publications

Simplifying the Modeling of Arbitrary Conditionals in Natural Language
Yinhan Lu
Eric Elmoznino
Leo Gagnon
Sarthak Mittal
Tejas Kasetty
Guillaume Lajoie
Causal Transformers model sequences through an autoregressive factorization of the joint distribution, which enables efficient left-to-right… (see more) decoding and conditional likelihood computation. However, they cannot tractably sample from or evaluate arbitrary conditionals -- e.g., a block of text conditioned on past and future tokens. Recent work aims to solve this problem through novel architectures, but they often lead to sub-optimal modeling of such conditionals and degraded generations. We propose Arbitrary Conditionals GPT (AC-GPT) which introduces a simple modification to standard causal Transformers to enable evaluating and sampling from arbitrary conditionals -- including past, future, and mixed contexts -- within a single forward pass. Unlike prior approaches, our method preserves the standard left-to-right ordering and next-token prediction objective essential for both strong performance and efficient training on natural language. Crucially, this compatibility allows existing LLMs to be fine-tuned for arbitrary conditioning. Our empirical results indicate that our method outperforms baselines on modeling arbitrary conditionals, without degrading standard left-to-right performance.
2026-06-11
arXiv (preprint)
doi.org
arxiv.org