Portrait de Marco Jiralerspong

Marco Jiralerspong

Doctorat - UdeM
Superviseur⋅e principal⋅e
Gauthier Gidel
Sujets de recherche
Apprentissage par renforcement
Apprentissage sur graphes
GFlowNets
Modèles génératifs
Théorie de l'apprentissage automatique
Site web
Google Scholar
GitHub

Publications

Feature Likelihood Score: Evaluating Generalization of Generative Models Using Samples
Marco Jiralerspong
Joey Bose
Gauthier Gidel
Deep generative models have demonstrated the ability to generate complex, high-dimensional, and photo-realistic data. However, a unified fr… (voir plus)amework for evaluating different generative modeling families remains a challenge. Indeed, likelihood-based metrics do not apply in many cases while pure sample-based metrics such as FID fail to capture known failure modes such as overfitting on training data. In this work, we introduce the Feature Likelihood Score (FLS), a parametric sample-based score that uses density estimation to quantitatively measure the quality/diversity of generated samples while taking into account overfitting. We empirically demonstrate the ability of FLS to identify specific overfitting problem cases, even when previously proposed metrics fail. We further perform an extensive experimental evaluation on various image datasets and model classes. Our results indicate that FLS matches intuitions of previous metrics, such as FID, while providing a more holistic evaluation of generative models that highlights models whose generalization abilities are under or overappreciated. Code for computing FLS is provided at https://github.com/marcojira/fls.
2023-01-01
arXiv.org (prépublication)
doi.org