Publications

Egalitarian Gradient Descent: A Simple Approach to Accelerated Grokking
Ali Saheb Pasand
Elvis Dopgima Dohmatob
Grokking is the phenomenon whereby, unlike the training performance, which peaks early in the training process, the test/generalization perf… (see more)ormance of a model stagnates over arbitrarily many epochs and then suddenly jumps to usually close to perfect levels. In practice, it is desirable to reduce the length of such plateaus, that is to make the learning process"grok"faster. In this work, we provide new insights into grokking. First, we show both empirically and theoretically that grokking can be induced by asymmetric speeds of (stochastic) gradient descent, along different principal (i.e singular directions) of the gradients. We then propose a simple modification that normalizes the gradients so that dynamics along all the principal directions evolves at exactly the same speed. Then, we establish that this modified method, which we call egalitarian gradient descent (EGD) and can be seen as a carefully modified form of natural gradient descent, groks much faster. In fact, in some cases the stagnation is completely removed. Finally, we empirically show that on classical arithmetic problems such as modular addition and sparse parity problem which this stagnation has been widely observed and intensively studied, that our proposed method eliminates the plateaus.
2025-09-30
arXiv (published)
doi.org
arxiv.org
Equivariant Geometric Scattering Networks via Vector Diffusion Wavelets
David R. Johnson
Rishabh Anand
Smita Krishnaswamy
Michael Perlmutter
2025-09-30
ArXiv (preprint)
doi.org
arxiv.org
Gaussian Embeddings: How JEPAs Secretly Learn Your Data Density
Randall Balestriero
Nicolas Ballas
Michael G. Rabbat
Yann Lecun
Joint Embedding Predictive Architectures (JEPAs) learn representations able to solve numerous downstream tasks out-of-the-box. JEPAs combine… (see more) two objectives: (i) a latent-space prediction term, i.e., the representation of a slightly perturbed sample must be predictable from the original sample's representation, and (ii) an anti-collapse term, i.e., not all samples should have the same representation. While (ii) is often considered as an obvious remedy to representation collapse, we uncover that JEPAs' anti-collapse term does much more--it provably estimates the data density. In short, any successfully trained JEPA can be used to get sample probabilities, e.g., for data curation, outlier detection, or simply for density estimation. Our theoretical finding is agnostic of the dataset and architecture used--in any case one can compute the learned probabilities of sample
2025-09-30
arXiv (published)
doi.org
arxiv.org
Improving GUI Grounding with Explicit Position-to-Coordinate Mapping
Suyuchen Wang
Tianyu Zhang
Ahmed Masry
Christopher Pal
Spandana Gella
Bang Liu
Perouz Taslakian
2025-09-30
arXiv (published)
doi.org
arxiv.org
Indirect Prompt Injections: Are Firewalls All You Need, or Stronger Benchmarks?
Rishika Bhagwatkar
Kevin Kasa
Abhay Puri
Gabriel Huang
Irina Rish
Graham W. Taylor
Krishnamurthy Dj Dvijotham
Alexandre Lacoste
AI agents are vulnerable to indirect prompt injection attacks, where malicious instructions embedded in external content or tool outputs cau… (see more)se unintended or harmful behavior. Inspired by the well-established concept of firewalls, we show that a simple, modular and model-agnostic defense operating at the agent--tool interface achieves perfect security (0% or the lowest possible attack success rate) with high utility (task success rate) across four public benchmarks: AgentDojo, Agent Security Bench, InjecAgent and tau-Bench, while achieving a state-of-the-art security-utility tradeoff compared to prior results. Specifically, we employ a defense based on two firewalls: a Tool-Input Firewall (Minimizer) and a Tool-Output Firewall (Sanitizer). Unlike prior complex approaches, this firewall defense makes minimal assumptions on the agent and can be deployed out-of-the-box, while maintaining strong performance without compromising utility. However, our analysis also reveals critical limitations in these existing benchmarks, including flawed success metrics, implementation bugs, and most importantly, weak attacks, hindering significant progress in the field. To foster more meaningful progress, we present targeted fixes to these issues for AgentDojo and Agent Security Bench while proposing best-practices for more robust benchmark design. Further, we demonstrate that although these firewalls push the state-of-the-art on existing benchmarks, it is still possible to bypass them in practice, underscoring the need to incorporate stronger attacks in security benchmarks. Overall, our work shows that existing agentic security benchmarks are easily saturated by a simple approach and highlights the need for stronger agentic security benchmarks with carefully chosen evaluation metrics and strong adaptive attacks.
2025-09-30
arXiv (published)
doi.org
arxiv.org
Intersecting perspectives: A participatory street review framework for urban inclusivity
Rashid Mushkani
Shin Koseki
2025-09-30
Habitat International (published)
doi.org
Just-in-time Episodic Feedback Hinter: Leveraging Offline Knowledge to Improve LLM Agents Adaptation
Hadi Nekoei
A. Jaiswal
Patrice Béchard
Oleh Shliazhko
Orlando Marquez Ayala
Mathieu Reymond
Massimo Caccia
Alexandre Drouin
A. Chandar
Alexandre Lacoste
2025-09-30
arXiv (published)
doi.org
arxiv.org
Learning What Matters: Steering Diffusion via Spectrally Anisotropic Forward Noise
Luca Scimeca
Thomas Jiralerspong
Berton Earnshaw
Jason Hartford
Yoshua Bengio
2025-09-30
arXiv (published)
doi.org
arxiv.org
Personalized Federated Fine-Tuning of Vision Foundation Models for Healthcare
Adam Tupper
Christian Gagné
2025-09-30
arXiv (published)
doi.org
arxiv.org
Predicting space use patterns of a territorial top predator: from individual movement decisions to Arctic fox space use
Frédéric Dulude-de Broin
Dominique Berteaux
Joël Bêty
Catherine Villeneuve
Alexis Grenier-Potvin
Andréanne Beardsell
Jeanne Clermont
Audrey Durand
Pierre Legagneux
2025-09-30
bioRxiv (preprint)
doi.org
"A Responsible Framework for Applying Artificial Intelligence on Medical Images and Signals at the Point of Care: The PACS-AI Platform [Canadian Journal of Cardiology Volume 40, Issue 10, October 2024, Pages 1828-1840]".
Pascal Thériault-Lauzier
Denis Corbin
Olivier Tastet
Élodie Labrecque Langlais
B. Taji
Guson Kang
A. Chong
Derek So
An Tang
J. W. Gichoya
A. Chandar
Pierre-Luc Deziel
Julie G Hussin
Samuel Kadoury
Robert Avram
2025-09-30
Canadian Journal of Cardiology (published)
doi.org
Similarity-based transfer learning with deep learning networks for accurate CRISPR-Cas9 off-target prediction.
Jérémy Charlier
Zeinab Sherkatghanad
Vladimir Makarenkov
Transfer learning has emerged as a powerful tool for enhancing predictive accuracy in complex tasks, particularly in scenarios where data is… (see more) limited or imbalanced. This study explores the use of similarity-based pre-evaluation as a methodology to identify optimal source datasets for transfer learning, addressing the dual challenge of efficient source-target dataset pairing and off-target prediction in CRISPR-Cas9, while existing transfer learning applications in the field of gene editing often lack a principled method for source dataset selection. We use cosine, Euclidean, and Manhattan distances to evaluate similarity between the source and target datasets used in our transfer learning experiments. Four deep learning network architectures, i.e. Multilayer Perceptron (MLP), Convolutional Neural Networks (CNNs), Feedforward Neural Networks (FNNs), and Recurrent Neural Networks (RNNs), and two traditional machine learning models, i.e. Logistic Regression (LR) and Random Forest (RF), were tested and compared in our simulations. The results suggest that similarity scores are reliable indicators for pre-selecting source datasets in CRISPR-Cas9 transfer learning experiments, with cosine distance proving to be a more effective dataset comparison metric than either Euclidean or Manhattan distances. An RNN-GRU, a 5-layer FNN, and two MLP variants provided the best overall prediction results in our simulations. By integrating similarity-based source pre-selection with machine learning outcomes, we propose a dual-layered framework that not only streamlines the transfer learning process but also significantly improves off-target prediction accuracy. The code and data used in this study are freely available at: https://github.com/dagrate/transferlearning_offtargets .
2025-09-30
PLoS Computational Biology (published)
doi.org