Portrait de Sihui Wei n'est pas disponible

Sihui Wei

sihui.wei@mila.quebec
Baccalauréat - McGill
Superviseur⋅e principal⋅e
Irina Rish
Sujets de recherche
Traitement du langage naturel

Publications

Toward a general understanding of neural representations learned by deep neural networks on group multiplications
Arthur Ayestas Hilgert
Sihui Wei
Doina Precup
Gabriela Moisescu-Pareja
Gavin McCracken
We study the neural representations learned by deep neural networks trained on alternating group multiplication, discovering that they are S… (voir plus)chreier coset graphs, a generalization of Cayley graphs. Previous works inspected neural representations learned from cyclic group multiplication and identified them as Cayley graphs. Since cyclic group multiplication is Abelian, all subgroups are normal, and Schreier coset graphs reduce to Cayley graphs in this case. This contribution makes a step toward finding a general theory of what neural representations are in networks learning group multiplications.
2026-07-15
Topology, Algebra, and Geometry in Data Science (présentation orale)
openreview.net
openreview.net
Deep neural networks divide and conquer dihedral multiplication
Sihui Wei
Gavin McCracken
Gabriela Moisescu-Pareja
Harley Wiltzer
Doina Precup
Irina Rish
Jonathan Love
We find multilayer perceptrons and transformers both universally learn an instantiation of the same divide-and-conquer algorithm that requir… (voir plus)es only a logarithmic number of neural representations to solve dihedral multiplication. Clustering neurons based on similar activation behaviour reveals remarkably clear structure: each neural representation corresponds to a Cayley graph. To our knowledge, this is the first work that fully characterizes and describes all neural representations that are learnable on a dataset, while prior work on group multiplications studied neuron-level behavior, or preliminarily investigated cluster behavior. Thus, we can understand the algorithm networks universally learn at three levels of abstraction: 1) Neurons activate on coset or approximate coset structure of the dihedral group. 2) Groups of neurons together form neural representations that act to divide the dataset into different subproblems, being Cayley graphs, where the equivalence class of the answer is computed. 3) The global algorithm then linearly combines each neural representation (subproblem) together at the logits. This work provides a deep case study and provides the community with a very well understood toy model for interpretability, as well as makes steps toward proving the conjecture that DNNs will divide and conquer all group multiplication tasks.
2025-12-31
International Conference on Machine Learning (Accept (regular))
openreview.net
openreview.net