Portrait de Nimrod De La Vega n'est pas disponible

Nimrod De La Vega

nimrod.delavega@gmail.com
Doctorat
Superviseur⋅e principal⋅e
Amir-massoud Farahmand
Sujets de recherche
Apprentissage par renforcement
Apprentissage profond
Optimisation
Systèmes dynamiques
Théorie de l'apprentissage automatique
Site web

Publications

SingularClip: Preventing Spectral Collapse to Maintain Plasticity in Continual and Reinforcement Learning
Tyler Kastner
Nimrod De La Vega
Amir-massoud Farahmand
Neural networks trained on nonstationary tasks frequently lose the ability to fit new targets, a phenomenon referred to as loss of plasticit… (voir plus)y. We identify a novel source of plasticity loss due to the growing anisotropy of weight matrices'singular values during training, and analyze this phenomenon both empirically and theoretically. To mitigate this issue, we introduce SingularClip, a procedure that periodically clips the singular values of all weight matrices. We show that SingularClip performs strongly against baselines across a range of tasks in both continual supervised learning and deep reinforcement learning.
2026-08-17
arXiv (prépublication)
doi.org
arxiv.org