Portrait de Charles-Etienne Joseph n'est pas disponible

Charles-Etienne Joseph

Maîtrise recherche - UdeM
Superviseur⋅e principal⋅e
Eugene Belilovsky
Co-supervisor
Guy Wolf
Sujets de recherche
Apprentissage de représentations
Apprentissage profond
Optimisation
Systèmes distribués
GitHub

Publications

Continual Pre-training of MoEs: How robust is your router?
Benjamin Therien
Charles-Etienne Joseph
Zain Sarwar
Ashwinee Panda
Anirban Das
Shi-Xiong Zhang
Stephen Rawls
Sambit Sahu
Eugene Belilovsky
Irina Rish
2025-03-06
ArXiv (prépublication)
arxiv.org
arxiv.org
Continual Pre-training of MoEs: How robust is your router?
Benjamin Therien
Charles-Etienne Joseph
Zain Sarwar
Ashwinee Panda
Anirban Das
Shi-Xiong Zhang
Stephen Rawls
Sambit Sahu
Eugene Belilovsky
Irina Rish
2025-03-01
arXiv (publié)
doi.org
arxiv.org
Meta-learning Optimizers for Communication-Efficient Learning
Charles-Etienne Joseph
Benjamin Therien
Abhinav Moudgil
Boris Knyazev
Eugene Belilovsky
2025-01-01
Trans. Mach. Learn. Res. (publié)
openreview.net
openreview.net
$\mu$LO: Compute-Efficient Meta-Generalization of Learned Optimizers
Benjamin Therien
Charles-Etienne Joseph
Boris Knyazev
Edouard Oyallon
Irina Rish
Eugene Belilovsky
2024-10-10
NeurIPS.cc/2024/Workshop/OPT (publié)
doi.org
openreview.net
μLO: Compute-Efficient Meta-Generalization of Learned Optimizers
Benjamin Therien
Charles-Etienne Joseph
Boris Knyazev
Edouard Oyallon
Irina Rish
Eugene Belilovsky
2024-05-31
ArXiv (prépublication)
doi.org
arxiv.org
μLO: Compute-Efficient Meta-Generalization of Learned Optimizers
Benjamin Therien
Charles-Etienne Joseph
Boris Knyazev
Edouard Oyallon
Irina Rish
Eugene Belilovsky
2024-05-31
ArXiv (prépublication)
doi.org
arxiv.org
Can We Learn Communication-Efficient Optimizers?
Charles-Etienne Joseph
Benjamin Therien
Abhinav Moudgil
Boris Knyazev
Eugene Belilovsky
2023-12-02
ArXiv (prépublication)
doi.org
arxiv.org
Learning Optimizers for Local SGD
Charles-Etienne Joseph
Benjamin Therien
Abhinav Moudgil
Boris Knyazev
Eugene Belilovsky
2023-10-27
NeurIPS.cc/2023/Workshop/Federated_Learning (poster)
openreview.net
openreview.net
Learning Optimizers for Local SGD
Charles-Etienne Joseph
Benjamin Therien
Abhinav Moudgil
Boris Knyazev
Eugene Belilovsky
Communication-efficient variants of SGD, specifically local SGD, have received a great deal of interest in recent years. These approaches co… (voir plus)mpute multiple gradient steps locally, that is on each worker, before averaging model parameters, helping relieve the critical communication bottleneck in distributed deep learning training. Although many variants of these approaches have been proposed, they can sometimes lag behind state-of-the-art optimizers for deep learning. In this work, we incorporate local optimizers that compute multiple updates into a learned optimization framework, allowing to meta-learn potentially more efficient local SGD algorithms. Our results demonstrate that local learned optimizers can substantially outperform local SGD and its sophisticated variants while maintaining their communication efficiency. We show that the learned optimizers can generalize to new datasets and architectures, demonstrating the potential of learned optimizers for improving communication-efficient distributed learning.
2000-01-01
(publié)
www.semanticscholar.org