Eric Elmoznino

Doctorat - UdeM

Superviseur⋅e principal⋅e

Yoshua Bengio

Co-supervisor

Guillaume Lajoie

Sujets de recherche

Apprentissage profond

Neurosciences computationnelles

Site web

Google Scholar

GitHub

Publications

Discrete, compositional, and symbolic representations through attractor dynamics

Andrew Nam

Chen Sun

Compositionality is an important feature of discrete symbolic systems, such as language and programs, as it enables them to have infinite ca… (voir plus)pacity despite a finite symbol set. It serves as a useful abstraction for reasoning in both cognitive science and in AI, yet the interface between continuous and symbolic processing is often imposed by fiat at the algorithmic level, such as by means of quantization or a softmax sampling step. In this work, we explore how discretization could be implemented in a more neurally plausible manner through the modeling of attractor dynamics that partition the continuous representation space into basins that correspond to sequences of symbols. Building on established work in attractor networks and introducing novel training methods, we show that imposing structure in the symbolic space can produce compositionality in the attractor-supported representation space of rich sensory inputs. Lastly, we argue that our model exhibits the process of an information bottleneck that is thought to play a role in conscious experience, decomposing the rich information of a sensory input into stable components encoding symbolic information.

2023-10-26

NeurIPS.cc/2023/Workshop/InfoCog (présentation orale)

doi.org

openreview.net

Learning Macro Variables with Auto-encoders

Dhanya Sridhar

Eric Elmoznino

Maitreyi Swaroop

2023-10-26

NeurIPS.cc/2023/Workshop/CRL (poster)

openreview.net