Portrait of Younesse Kaddar is unavailable

Younesse Kaddar

younesse.kaddar@mila.quebec
Research Intern - Université de Montréal
Supervisor
Yoshua Bengio
Website
Github

Publications

Amortizing intractable inference in large language models
Edward J Hu
Moksh J. Jain
Eric Elmoznino
Younesse Kaddar
Guillaume Lajoie
Yoshua Bengio
Nikolay Malkin
Autoregressive large language models (LLMs) compress knowledge from their training data through next-token conditional distributions. This l… (see more)imits tractable querying of this knowledge to start-to-end autoregressive sampling. However, many tasks of interest -- including sequence continuation, infilling, and other forms of constrained generation -- involve sampling from intractable posterior distributions. We address this limitation by using amortized Bayesian inference to sample from these intractable posteriors. Such amortization is algorithmically achieved by fine-tuning LLMs via diversity-seeking reinforcement learning algorithms: generative flow networks (GFlowNets). We empirically demonstrate that this distribution-matching paradigm of LLM fine-tuning can serve as an effective alternative to maximum-likelihood training and reward-maximizing policy optimization. As an important application, we interpret chain-of-thought reasoning as a latent variable modeling problem and demonstrate that our approach enables data-efficient adaptation of LLMs to tasks that require multi-step rationalization and tool use.
2024-01-16
ICLR.cc/2024/Conference (oral)
doi.org
openreview.net