Portrait de Prasanna Parthasarathi n'est pas disponible

Prasanna Parthasarathi

Alumni

Publications

Memory Augmented Optimizers for Deep Learning
Paul-Aymeric McRae
Prasanna Parthasarathi
Mahmoud Assran
Sarath Chandar
Popular approaches for minimizing loss in data-driven learning often involve an abstraction or an explicit retention of the history of gradi… (voir plus)ents for efficient parameter updates. The aggregated history of gradients nudges the parameter updates in the right direction even when the gradients at any given step are not informative. Although the history of gradients summarized in meta-parameters or explicitly stored in memory has been shown effective in theory and practice, the question of whether
2022-01-27
ICLR.cc/2022/Conference (poster)
doi.org
openreview.net
Detecting Languages Unintelligible to Multilingual Models through Local Structure Probes
Louis Clouatre
Prasanna Parthasarathi
Amal Zouaq
Sarath Chandar
Providing better language tools for low-resource and endangered languages is imperative for equitable growth. Recent progress with massively… (voir plus) multilingual pretrained models has proven surprisingly effective at performing zero-shot transfer to a wide variety of languages. However, this transfer is not universal, with many languages not currently understood by multilingual approaches. It is estimated that only 72 languages possess a "small set of labeled datasets" on which we could test a model's performance, the vast majority of languages not having the resources available to simply evaluate performances on. In this work, we attempt to clarify which languages do and do not currently benefit from such transfer. To that end, we develop a general approach that requires only unlabelled text to detect which languages are not well understood by a cross-lingual model. Our approach is derived from the hypothesis that if a model's understanding is insensitive to perturbations to text in a language, it is likely to have a limited understanding of that language. We construct a cross-lingual sentence similarity task to evaluate our approach empirically on 350, primarily low-resource, languages.
2021-12-31
EMNLP (Findings) (publié)
doi.org
arxiv.org
Local Structure Matters Most in Most Languages
Louis Clouatre
Prasanna Parthasarathi
Amal Zouaq
A. Chandar
2021-12-31
AACL/IJCNLP (2) (publié)
doi.org
arxiv.org
A Brief Study on the Effects of Training Generative Dialogue Models with a Semantic loss
Prasanna Parthasarathi
Mohamed Abdelsalam
Joelle Pineau
Sarath Chandar
Neural models trained for next utterance generation in dialogue task learn to mimic the n-gram sequences in the training set with training o… (voir plus)bjectives like negative log-likelihood (NLL) or cross-entropy. Such commonly used training objectives do not foster generating alternate responses to a context. But, the effects of minimizing an alternate training objective that fosters a model to generate alternate response and score it on semantic similarity has not been well studied. We hypothesize that a language generation model can improve on its diversity by learning to generate alternate text during training and minimizing a semantic loss as an auxiliary objective. We explore this idea on two different sized data sets on the task of next utterance generation in goal oriented dialogues. We make two observations (1) minimizing a semantic objective improved diversity in responses in the smaller data set (Frames) but only as-good-as minimizing the NLL in the larger data set (MultiWoZ) (2) large language model embeddings can be more useful as a semantic loss objective than as initialization for token embeddings.
2021-06-30
Proceedings of the 22nd Annual Meeting of the Special Interest Group on Discourse and Dialogue (publié)
doi.org
arxiv.org