Portrait de Gabriela Moisescu-Pareja n'est pas disponible

Gabriela Moisescu-Pareja

Collaborateur·rice de recherche - McGill
Superviseur⋅e principal⋅e
Doina Precup
Co-supervisor
Irina Rish
Sujets de recherche
Apprentissage de représentations
Apprentissage par renforcement
Apprentissage profond
Modélisation moléculaire
Réseaux de neurones en graphes
Théorie de l'apprentissage automatique

Publications

The Geometry and Topology of Circuits: the Manifolds of Modular Addition
Gabriela Moisescu-Pareja
Gavin McCracken
Harley Wiltzer
Vincent Létourneau
Colin Daniels
Doina Precup
Jonathan Love
The Clock and Pizza interpretations, associated with architectures differing in either uniform or learnable attention, were introduced to ar… (voir plus)gue that different architectural designs can yield distinct circuits for modular addition. In this work, we show that this is not the case, and that both the uniform and trainable attention architectures implement the same algorithm via topologically and geometrically equivalent representations. Our methodology goes beyond the interpretation of individual neurons and weights. Instead, we identify all of the neurons corresponding to each learned representation and then study the collective group of neurons as one entity. This method reveals that each learned representation is a manifold that we can study utilizing tools from topology. Based on this insight, we can statistically analyze the learned representations across hundreds of circuits to demonstrate the similarity between learned modular addition circuits that arise naturally from common deep learning paradigms.
2025-12-31
International Conference on Learning Representations (Accept (Poster))
openreview.net
openreview.net
On the geometry and topology of representations: the manifolds of modular addition
Gabriela Moisescu-Pareja
Gavin McCracken
Harley Wiltzer
Vincent Létourneau
Colin Daniels
Doina Precup
Jonathan Love
The Clock and Pizza interpretations, associated with architectures differing in either uniform or learnable attention, were introduced to ar… (voir plus)gue that different architectural designs can yield distinct circuits for modular addition. In this work, we show that this is not the case, and that both uniform attention and trainable attention architectures implement the same algorithm via topologically and geometrically equivalent representations. Our methodology goes beyond the interpretation of individual neurons and weights. Instead, we identify all of the neurons corresponding to each learned representation and then study the collective group of neurons as one entity. This method reveals that each learned representation is a manifold that we can study utilizing tools from topology. Based on this insight, we can statistically analyze the learned representations across hundreds of circuits to demonstrate the similarity between learned modular addition circuits that arise naturally from common deep learning paradigms.
2025-12-30
arXiv (prépublication)
doi.org
openreview.net
The Geometry and Topology of Modular Addition Representations
Gabriela Moisescu-Pareja
Gavin McCracken
Harley Wiltzer
Colin Daniels
Vincent Létourneau
Doina Precup
Jonathan Love
The Clock and Pizza interpretations, associated with neural architectures differing in either uniform or learnable attention, were introduce… (voir plus)d to argue that different architectural designs can yield distinct circuits for modular addition. Applying geometric and topological analyses to learned representations, we show that this is not the case: Clock and Pizza circuits are topologically and geometrically equivalent and are thus equivalent representations.
2025-11-12
TAG-DS/2025/Conference (poster)
openreview.net
openreview.net
Unifying Mechanistic Interpretations of Neural Networks Trained on Modular Addition
Gabriela Moisescu-Pareja
Gavin McCracken
Vincent Létourneau
Doina Precup
Jonathan Love
2025-09-21
NeurIPS.cc/2025/Workshop/WiML (publié)
openreview.net
openreview.net
Uncovering a Universal Abstract Algorithm for Modular Addition in Neural Networks
Gavin McCracken
Gabriela Moisescu-Pareja
Vincent Létourneau
Doina Precup
Jonathan Love
We propose a testable universality hypothesis, asserting that seemingly disparate neural network solutions observed in the simple task of mo… (voir plus)dular addition are unified under a common abstract algorithm. While prior work interpreted variations in neuron-level representations as evidence for distinct algorithms, we demonstrate - through multi-level analyses spanning neurons, neuron clusters, and entire networks - that multilayer perceptrons and transformers universally implement the abstract algorithm we call the approximate Chinese Remainder Theorem. Crucially, we introduce approximate cosets and show that neurons activate exclusively on them. Furthermore, our theory works for deep neural networks (DNNs). It predicts that universally learned solutions in DNNs with trainable embeddings or more than one hidden layer require only O(log n) features, a result we empirically confirm. This work thus provides the first theory-backed interpretation of multilayer networks solving modular addition. It advances generalizable interpretability and opens a testable universality hypothesis for group multiplication beyond modular addition.
2025-09-17
NeurIPS.cc/2025/Conference (poster)
doi.org
openreview.net
Towards Foundational Models for Molecular Learning on Large-Scale Multi-Task Datasets
Dominique Beaini
Shenyang Huang
Joao Alex Cunha
Zhiyi Li
Gabriela Moisescu-Pareja
Oleksandr Dymov
Samuel Maddrell-Mander
Callum McLean
Frederik Wenkel
Luis Müller
Jama Hussein Mohamud
Ali Parviz
Michael Craig
Michał Koziarski
Jiarui Lu
Zhaocheng Zhu
Cristian Gabellini
Kerstin Klasers
Josef Dean
Cas Wognum … (voir 15 de plus)
Maciej Sypetkowski
Guillaume Rabusseau
Reihaneh Rabbany
Jian Tang
Christopher Morris
Ioannis Koutis
Mirco Ravanelli
Guy Wolf
Prudencio Tossou
Hadrien Mary
Therence Bois
Andrew Fitzgibbon
Błażej Banaszewski
Chad Martin
Dominic Masters
Recently, pre-trained foundation models have shown significant advancements in multiple fields. However, the lack of datasets with labeled f… (voir plus)eatures and codebases has hindered the development of a supervised foundation model for molecular tasks. Here, we have carefully curated seven datasets specifically tailored for node- and graph-level prediction tasks to facilitate supervised learning on molecules. Moreover, to support the development of multi-task learning on our proposed datasets, we created the Graphium graph machine learning library. Our dataset collection encompasses two distinct categories. Firstly, the TOYMIX category modifies three small existing datasets with additional data for multi-task learning. Secondly, the LARGEMIX category includes four large-scale datasets with 344M graph-level data points and 409M node-level data points from ∼5M unique molecules. Finally, the ultra-large dataset contains 2,210M graph-level data points and 2,031M node-level data points coming from 86M molecules. Hence our datasets represent an order of magnitude increase in data volume compared to other 2D-GNN datasets. In addition, recognizing that molecule-related tasks often span multiple levels, we have designed our library to explicitly support multi-tasking, offering a diverse range of multi-level representations, i.e., representations at the graph, node, edge, and node-pair level. We equipped the library with an extensive collection of models and features to cover different levels of molecule analysis. By combining our curated datasets with this versatile library, we aim to accelerate the development of molecule foundation models. Datasets and code are available at https://github.com/datamol-io/graphium.
2024-01-15
ICLR.cc/2024/Conference (poster)
doi.org
openreview.net