Portrait de Eric Elmoznino n'est pas disponible

Eric Elmoznino

eric.elmoznino@mila.quebec
Doctorat - Université de Montréal
Superviseur⋅e principal⋅e
Yoshua Bengio
Co-superviseur⋅e
Guillaume Lajoie
Site web
Github
Google Scholar

Publications

Sources of richness and ineffability for phenomenally conscious states
Xu Ji
Eric Elmoznino
George Deane
Axel Constant
Guillaume Dumas
Guillaume Lajoie
Jonathan Simon
Yoshua Bengio
2024-03-01
Neuroscience of Consciousness (publié)
doi.org
arxiv.org
Amortizing intractable inference in large language models
Edward J Hu
Moksh J. Jain
Eric Elmoznino
Younesse Kaddar
Guillaume Lajoie
Yoshua Bengio
Nikolay Malkin
Autoregressive large language models (LLMs) compress knowledge from their training data through next-token conditional distributions. This l… (voir plus)imits tractable querying of this knowledge to start-to-end autoregressive sampling. However, many tasks of interest -- including sequence continuation, infilling, and other forms of constrained generation -- involve sampling from intractable posterior distributions. We address this limitation by using amortized Bayesian inference to sample from these intractable posteriors. Such amortization is algorithmically achieved by fine-tuning LLMs via diversity-seeking reinforcement learning algorithms: generative flow networks (GFlowNets). We empirically demonstrate that this distribution-matching paradigm of LLM fine-tuning can serve as an effective alternative to maximum-likelihood training and reward-maximizing policy optimization. As an important application, we interpret chain-of-thought reasoning as a latent variable modeling problem and demonstrate that our approach enables data-efficient adaptation of LLMs to tasks that require multi-step rationalization and tool use.
2024-01-16
ICLR.cc/2024/Conference (présentation orale)
doi.org
openreview.net
Discrete, compositional, and symbolic representations through attractor dynamics
Andrew Nam
Eric Elmoznino
Nikolay Malkin
Chen Sun
Yoshua Bengio
Guillaume Lajoie
Compositionality is an important feature of discrete symbolic systems, such as language and programs, as it enables them to have infinite ca… (voir plus)pacity despite a finite symbol set. It serves as a useful abstraction for reasoning in both cognitive science and in AI, yet the interface between continuous and symbolic processing is often imposed by fiat at the algorithmic level, such as by means of quantization or a softmax sampling step. In this work, we explore how discretization could be implemented in a more neurally plausible manner through the modeling of attractor dynamics that partition the continuous representation space into basins that correspond to sequences of symbols. Building on established work in attractor networks and introducing novel training methods, we show that imposing structure in the symbolic space can produce compositionality in the attractor-supported representation space of rich sensory inputs. Lastly, we argue that our model exhibits the process of an information bottleneck that is thought to play a role in conscious experience, decomposing the rich information of a sensory input into stable components encoding symbolic information.
2023-10-27
NeurIPS.cc/2023/Workshop/InfoCog (présentation orale)
doi.org
openreview.net
Learning Macro Variables with Auto-encoders
Dhanya Sridhar
Eric Elmoznino
Maitreyi Swaroop
2023-10-27
NeurIPS.cc/2023/Workshop/CRL (poster)
openreview.net
openreview.net
Consciousness in Artificial Intelligence: Insights from the Science of Consciousness
Patrick Mark Butlin
R. Long
Eric Elmoznino
Yoshua Bengio
Jonathan C. P. Birch
Axel Constant
George Deane
S. Fleming
C. Frith
Xuanxiu Ji
Ryota Kanai
C. Klein
Grace W. Lindsay
Matthias Michel
Liad Mudrik
Megan A. K. Peters
Eric Schwitzgebel
Jonathan Simon
Rufin Vanrullen
2023-08-17
ArXiv (prépublication)
doi.org
arxiv.org