Portrait of Nimrod De La Vega is unavailable

Nimrod De La Vega

nimrod.delavega@gmail.com
PhD
Supervisor
Amir-massoud Farahmand
Research Topics
Deep Learning
Dynamical Systems
Machine Learning Theory
Optimization
Reinforcement Learning
Website

Publications

SingularClip: Preventing Spectral Collapse to Maintain Plasticity in Continual and Reinforcement Learning
Tyler Kastner
Nimrod De La Vega
Amir-massoud Farahmand
Neural networks trained on nonstationary tasks frequently lose the ability to fit new targets, a phenomenon referred to as loss of plasticit… (see more)y. We identify a novel source of plasticity loss due to the growing anisotropy of weight matrices'singular values during training, and analyze this phenomenon both empirically and theoretically. To mitigate this issue, we introduce SingularClip, a procedure that periodically clips the singular values of all weight matrices. We show that SingularClip performs strongly against baselines across a range of tasks in both continual supervised learning and deep reinforcement learning.
2026-08-17
arXiv (preprint)
doi.org
arxiv.org