Portrait de Gintare Karolina Dziugaite

Gintare Karolina Dziugaite

Membre industriel associé
Professeure associée, McGill University, École d'informatique
Chercheuse scientifique senior, Google DeepMind
Site web
Google Scholar
LinkedIn

Biographie

Gintare Karolina Dziugaite est chercheuse scientifique senior chez Google DeepMind, à Toronto, et professeure associée à l'École d'informatique de l'Université McGill. Avant de se joindre à Google, elle a dirigé le programme Trustworthy AI chez Element AI / ServiceNow. Ses recherches combinent des approches théoriques et empiriques visant à comprendre l'apprentissage profond.

Gintare Karolina Dziugaite est bien connue pour ses travaux sur la rareté des réseaux et des données, le développement d'algorithmes et la découverte des effets sur la généralisation et d'autres mesures. Elle a été la première à étudier la connectivité des modes linéaires, en les reliant d'abord à l'existence des billets de loterie, puis aux paysages de pertes et au mécanisme d'élagage itératif de la magnitude. Ses recherches portent également sur la compréhension de la généralisation dans l'apprentissage profond et, plus généralement, sur le développement de méthodes fondées sur la théorie de l'information pour l'étude de la généralisation. Ses travaux les plus récents s’intéressent à l'élimination de l'influence des données sur le modèle (désapprentissage).

Mme Dziugaite a obtenu un doctorat en apprentissage automatique de l'Université de Cambridge, sous la direction de Zoubin Ghahramani. Elle a étudié les mathématiques à l'Université de Warwick et a suivi la partie III des mathématiques à l'Université de Cambridge, où elle a obtenu un Master of Advanced Studies (M.A.St.) en mathématiques. Elle a participé à plusieurs programmes de longue durée à l'Institute for Advanced Study de l’Université Princeton (New Jersey) et au Simons Institute for the Theory of Computing de l'Université de Berkeley.

Publications

When Majorities Prevent Learning: Eliminating Bias to Improve Worst-group and Out-of-distribution Generalization
Yu Yang
Gintare Karolina Dziugaite
Baharan Mirzasoleiman
Modern neural networks trained on large datasets have achieved state-of-the-art (in-distribution) generalization performance on various task… (voir plus)s. However, their good generalization performance has been shown to be contributed largely to overfitting spurious biases in large datasets. This is evident by the poor generalization performance of such models on minorities and out-of-distribution data. To alleviate this issue, subsampling the majority groups has been shown to be very effective. However, it is not clear how to find the subgroups (e.g. within a class) in large real-world datasets. Besides, naively subsampling the majority groups can entirely deplete some of their smaller sub-populations and drastically harm the in-distribution performance. Here, we show that tracking gradient trajectories of examples in initial epochs allows for finding large subpopulations of data points. We leverage this observation and propose an importance sampling method that is biased towards selecting smaller subpopulations, and eliminates bias in the large subpopulations. Our experiments confirm the effectiveness of our approach in eliminating spurious biases and learning higher-quality models with superior in- and out-of-distribution performance on various datasets.
2023-02-01
ICLR.cc/2023/Conference (rejected)
openreview.net
openreview.net
Stochastic Neural Network with Kronecker Flow
Chin-Wei Huang
Ahmed Touati
Pascal Vincent
Gintare Karolina Dziugaite
Alexandre Lacoste
Aaron Courville
Recent advances in variational inference enable the modelling of highly structured joint distributions, but are limited in their capacity to… (voir plus) scale to the high-dimensional setting of stochastic neural networks. This limitation motivates a need for scalable parameterizations of the noise generation process, in a manner that adequately captures the dependencies among the various parameters. In this work, we address this need and present the Kronecker Flow, a generalization of the Kronecker product to invertible mappings designed for stochastic neural networks. We apply our method to variational Bayesian neural networks on predictive tasks, PAC-Bayes generalization bound estimation, and approximate Thompson sampling in contextual bandits. In all setups, our methods prove to be competitive with existing methods and better than the baselines.
2020-06-03
Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics (publié)
proceedings.mlr.press
arxiv.org
In Search of Robust Measures of Generalization
Gintare Karolina Dziugaite
Alexandre Drouin
Brady Neal
Nitarshan Rajkumar
Ethan Caballero
Linbo Wang
Ioannis Mitliagkas
Daniel M. Roy
One of the principal scientific challenges in deep learning is explaining generalization, i.e., why the particular way the community now tra… (voir plus)ins networks to achieve small training error also leads to small error on held-out data from the same population. It is widely appreciated that some worst-case theories -- such as those based on the VC dimension of the class of predictors induced by modern neural network architectures -- are unable to explain empirical performance. A large volume of work aims to close this gap, primarily by developing bounds on generalization error, optimization error, and excess risk. When evaluated empirically, however, most of these bounds are numerically vacuous. Focusing on generalization bounds, this work addresses the question of how to evaluate such bounds empirically. Jiang et al. (2020) recently described a large-scale empirical study aimed at uncovering potential causal relationships between bounds/measures and generalization. Building on their study, we highlight where their proposed methods can obscure failures and successes of generalization measures in explaining generalization. We argue that generalization measures should instead be evaluated within the framework of distributional robustness.
arxiv.org
Stochastic Neural Network with Kronecker Flow
Chin-Wei Huang
Ahmed Touati
Pascal Vincent
Gintare Karolina Dziugaite
Alexandre Lacoste
Aaron Courville
Recent advances in variational inference enable the modelling of highly structured joint distributions, but are limited in their capacity to… (voir plus) scale to the high-dimensional setting of stochastic neural networks. This limitation motivates a need for scalable parameterizations of the noise generation process, in a manner that adequately captures the dependencies among the various parameters. In this work, we address this need and present the Kronecker Flow, a generalization of the Kronecker product to invertible mappings designed for stochastic neural networks. We apply our method to variational Bayesian neural networks on predictive tasks, PAC-Bayes generalization bound estimation, and approximate Thompson sampling in contextual bandits. In all setups, our methods prove to be competitive with existing methods and better than the baselines.
2019-06-10
ArXiv (preprint)
arxiv.org
arxiv.org