Portrait de Theodore Glavas n'est pas disponible

Theodore Glavas

Maîtrise recherche - McGill
Superviseur⋅e principal⋅e
Sujets de recherche
Apprentissage profond
Grands modèles de langage (LLM)
IA et durabilité
Traitement du langage naturel

Publications

Is the acquisition worth the cost? Surrogate losses for Consistent Two-stage Classifiers
Dynamic layer selection in decoder-only transformers
Joud Chataoui
Florence Regol
Wassim Jabbour
Boris N. Oreshkin
The vast size of Large Language Models (LLMs) has prompted a search to optimize inference. One effective approach is dynamic inference, whic… (voir plus)h adapts the architecture to the sample-at-hand to reduce the overall computational cost. We empirically examine two common dynamic inference methods for natural language generation (NLG): layer skipping and early exiting. We find that a pre-trained decoder-only model is significantly more robust to layer removal via layer skipping, as opposed to early exit. We demonstrate the difficulty of using hidden state information to adapt computation on a per-token basis for layer skipping. Finally, we show that dynamic computation allocation on a per-sequence basis holds promise for significant efficiency gains by constructing an oracle controller. Remarkably, we find that there exists an allocation which achieves equal performance to the full model using only 23.3% of its layers on average.