Portrait of (Rex) Devon Hjelm

(Rex) Devon Hjelm

Affiliate Member
Research Scientist, Apple MLR
Research Topics
Causality
Deep Learning
Generative Models
Information Theory
Online Learning
Probabilistic Models
Reasoning
Reinforcement Learning
Representation Learning
Google Scholar

Publications

Deep Graph Infomax
Petar Veličković
William Fedus
William L. Hamilton
Pietro Lio
Yoshua Bengio
(Rex) Devon Hjelm
We present Deep Graph Infomax (DGI), a general approach for learning node representations within graph-structured data in an unsupervised ma… (see more)nner. DGI relies on maximizing mutual information between patch representations and corresponding high-level summaries of graphs---both derived using established graph convolutional network architectures. The learnt patch representations summarize subgraphs centered around nodes of interest, and can thus be reused for downstream node-wise learning tasks. In contrast to most prior approaches to unsupervised learning with GCNs, DGI does not rely on random walk objectives, and is readily applicable to both transductive and inductive learning setups. We demonstrate competitive performance on a variety of node classification benchmarks, which at times even exceeds the performance of supervised learning.
2018-09-27
ArXiv (preprint)
arxiv.org
arxiv.org
Deep Graph Infomax
Petar Veličković
William Fedus
William L. Hamilton
Pietro Lio
Yoshua Bengio
(Rex) Devon Hjelm
We present Deep Graph Infomax (DGI), a general approach for learning node representations within graph-structured data in an unsupervised ma… (see more)nner. DGI relies on maximizing mutual information between patch representations and corresponding high-level summaries of graphs---both derived using established graph convolutional network architectures. The learnt patch representations summarize subgraphs centered around nodes of interest, and can thus be reused for downstream node-wise learning tasks. In contrast to most prior approaches to unsupervised learning with GCNs, DGI does not rely on random walk objectives, and is readily applicable to both transductive and inductive learning setups. We demonstrate competitive performance on a variety of node classification benchmarks, which at times even exceeds the performance of supervised learning.
2018-09-27
ArXiv (preprint)
arxiv.org
arxiv.org
Deep Graph Infomax
Petar Veličković
William Fedus
William L. Hamilton
Pietro Lio
Yoshua Bengio
(Rex) Devon Hjelm
2018-09-27
ArXiv (preprint)
arxiv.org
arxiv.org
Deep Graph Infomax
Petar Veličković
William Fedus
William L. Hamilton
Pietro Lio
Yoshua Bengio
(Rex) Devon Hjelm
2018-09-27
ArXiv (preprint)
arxiv.org
arxiv.org
Deep Graph Infomax
Petar Veličković
William Fedus
William L. Hamilton
Pietro Lio
Yoshua Bengio
(Rex) Devon Hjelm
We present Deep Graph Infomax (DGI), a general approach for learning node representations within graph-structured data in an unsupervised ma… (see more)nner. DGI relies on maximizing mutual information between patch representations and corresponding high-level summaries of graphs---both derived using established graph convolutional network architectures. The learnt patch representations summarize subgraphs centered around nodes of interest, and can thus be reused for downstream node-wise learning tasks. In contrast to most prior approaches to unsupervised learning with GCNs, DGI does not rely on random walk objectives, and is readily applicable to both transductive and inductive learning setups. We demonstrate competitive performance on a variety of node classification benchmarks, which at times even exceeds the performance of supervised learning.
2018-09-27
ArXiv (preprint)
arxiv.org
arxiv.org
Deep Graph Infomax
Petar Veličković
William Fedus
William L. Hamilton
Pietro Lio
Yoshua Bengio
(Rex) Devon Hjelm
We present Deep Graph Infomax (DGI), a general approach for learning node representations within graph-structured data in an unsupervised ma… (see more)nner. DGI relies on maximizing mutual information between patch representations and corresponding high-level summaries of graphs---both derived using established graph convolutional network architectures. The learnt patch representations summarize subgraphs centered around nodes of interest, and can thus be reused for downstream node-wise learning tasks. In contrast to most prior approaches to unsupervised learning with GCNs, DGI does not rely on random walk objectives, and is readily applicable to both transductive and inductive learning setups. We demonstrate competitive performance on a variety of node classification benchmarks, which at times even exceeds the performance of supervised learning.
2018-09-27
ArXiv (preprint)
arxiv.org
arxiv.org
Deep Graph Infomax
Petar Veličković
William Fedus
William L. Hamilton
Pietro Lio
Yoshua Bengio
(Rex) Devon Hjelm
2018-09-27
ArXiv (preprint)
arxiv.org
arxiv.org
Deep Graph Infomax
Petar Veličković
William Fedus
William L. Hamilton
Pietro Lio
Yoshua Bengio
(Rex) Devon Hjelm
We present Deep Graph Infomax (DGI), a general approach for learning node representations within graph-structured data in an unsupervised ma… (see more)nner. DGI relies on maximizing mutual information between patch representations and corresponding high-level summaries of graphs---both derived using established graph convolutional network architectures. The learnt patch representations summarize subgraphs centered around nodes of interest, and can thus be reused for downstream node-wise learning tasks. In contrast to most prior approaches to unsupervised learning with GCNs, DGI does not rely on random walk objectives, and is readily applicable to both transductive and inductive learning setups. We demonstrate competitive performance on a variety of node classification benchmarks, which at times even exceeds the performance of supervised learning.
2018-09-27
ArXiv (preprint)
arxiv.org
arxiv.org
Deep Graph Infomax
Petar Veličković
William Fedus
William L. Hamilton
Pietro Lio
Yoshua Bengio
(Rex) Devon Hjelm
2018-09-27
ArXiv (preprint)
arxiv.org
arxiv.org
Deep Graph Infomax
Petar Veličković
William Fedus
William L. Hamilton
Pietro Lio
Yoshua Bengio
(Rex) Devon Hjelm
2018-09-27
ArXiv (preprint)
arxiv.org
arxiv.org
Mutual Information Neural Estimation
Ishmael Belghazi
Aristide Baratin
Sai Rajeswar
Sherjil Ozair
Yoshua Bengio
(Rex) Devon Hjelm
Aaron Courville
We argue that the estimation of mutual information between high dimensional continuous random variables can be achieved by gradient descent … (see more)over neural networks. We present a Mutual Information Neural Estimator (MINE) that is linearly scalable in dimensionality as well as in sample size, trainable through back-prop, and strongly consistent. We present a handful of applications on which MINE can be used to minimize or maximize mutual information. We apply MINE to improve adversarially trained generative models. We also use MINE to implement Information Bottleneck, applying it to supervised classification; our results demonstrate substantial improvement in flexibility and performance in these settings.
2018-07-03
Proceedings of the 35th International Conference on Machine Learning (published)
proceedings.mlr.press
arxiv.org
MINE: Mutual Information Neural Estimation
Ishmael Belghazi
Sai Rajeswar
Aristide Baratin
(Rex) Devon Hjelm
Aaron Courville
This paper presents a Mutual Information Neural Estimator (MINE) that is linearly scalable in dimensionality as well as in sample size. MINE… (see more) is back-propable and we prove that it is strongly consistent. We illustrate a handful of applications in which MINE is succesfully applied to enhance the property of generative models in both unsupervised and supervised settings. We apply our framework to estimate the information bottleneck, and apply it in tasks related to supervised classification problems. Our results demonstrate substantial added flexibility and improvement in these settings.
2018-05-04
ICLR.cc/2018/Conference (unknown)
openreview.net
openreview.net