Portrait de Karan Grewal n'est pas disponible

Karan Grewal

Alumni

Publications

Learning deep representations by mutual information estimation and maximization
R Devon Hjelm
Alex Fedorov
Samuel Lavoie-Marchildon
Karan Grewal
Adam Trischler
Phil Bachman
Yoshua Bengio
This work investigates unsupervised learning of representations by maximizing mutual information between an input and the output of a deep n… (voir plus)eural network encoder. Importantly, we show that structure matters: incorporating knowledge about locality in the input into the objective can significantly improve a representation’s suitability for downstream tasks. We further control characteristics of the representation by matching to a prior distribution adversarially. Our method, which we call Deep InfoMax (DIM), outperforms a number of popular unsupervised learning methods and compares favorably with fully-supervised learning on several classification tasks in with some standard architectures. DIM opens new avenues for unsupervised learning of representations and is an important step towards flexible formulations of representation learning objectives for specific end-goals.
2018-12-31
ICLR (publié)
openreview.net
openreview.net
Variance Regularizing Adversarial Learning
Karan Grewal
R Devon Hjelm
Yoshua Bengio
We study how, in generative adversarial networks, variance in the discriminator's output affects the generator's ability to learn the data d… (voir plus)istribution. In particular, we contrast the results from various well-known techniques for training GANs when the discriminator is near-optimal and updated multiple times per update to the generator. As an alternative, we propose an additional method to train GANs by explicitly modeling the discriminator's output as a bi-modal Gaussian distribution over the real/fake indicator variables. In order to do this, we train the Gaussian classifier to match the target bi-modal distribution implicitly through meta-adversarial training. We observe that our new method, when trained together with a strong discriminator, provides meaningful, non-vanishing gradients.
2017-07-01
ArXiv (prépublication)
openreview.net
openreview.net