Portrait of Chin-wei Huang is unavailable

Chin-wei Huang

Alumni

Publications

Learning to Dequantise with Truncated Flows
Shawn Tan
Chin-wei Huang
Alessandro Sordoni
Aaron Courville
Dequantisation is a general technique used for transforming data described by a discrete random variable x into a continuous (latent) random… (see more) variable z, for the purpose of it being modeled by likelihood-based density models. Dequantisation was first introduced in the context of ordinal data, such as image pixel values. However, when the data is categorical, the dequantisation scheme is not obvious. We learn such a dequantisation scheme q(z|x), using variational inference with TRUncated FLows (TRUFL) — a novel flow-based model that allows the dequantiser to have a learnable truncated support. Unlike previous work, the TRUFL dequantiser is (i) capable of embedding the data losslessly in certain cases, since the truncation allows the conditional distributions q(z|x) to have non-overlapping bounded supports, while being (ii) trainable with back-propagation. Addtionally, since the support of the marginal q(z) is bounded and the support of prior p(z) is not, we propose to renormalise the prior distribution over the support of q(z). We derive a lower bound for training, and propose a rejection sampling scheme to account for the invalid samples. Experimentally, we benchmark TRUFL on constrained generation tasks, and find that it outperforms prior approaches. In addition, we find that rejection sampling results in higher validity for the constrained problems.
2022-01-01
International Conference on Learning Representations (published)
openreview.net
openreview.net
Riemannian Diffusion Models
Chin-wei Huang
Milad Aghajohari
Joey Bose
Prakash Panangaden
Aaron Courville
Diffusion models are recent state-of-the-art methods for image generation and likelihood estimation. In this work, we generalize continuous-… (see more)time diffusion models to arbitrary Riemannian manifolds and derive a variational framework for likelihood estimation. Computationally, we propose new methods for computing the Riemannian divergence which is needed for likelihood estimation. Moreover, in generalizing the Euclidean case, we prove that maximizing this variational lower-bound is equivalent to Riemannian score matching. Empirically, we demonstrate the expressive power of Riemannian diffusion models on a wide spectrum of smooth manifolds, such as spheres, tori, hyperboloids, and orthogonal groups. Our proposed method achieves new state-of-the-art likelihoods on all benchmarks.
openreview.net
Problèmes associés au déploiement des modèles fondés sur l’apprentissage machine en santé
Joseph Paul Cohen
Tianshi Cao
Joseph D Viviano
Chin-wei Huang
Michael Fralick
Marzyeh Ghassemi
Muhammad Mamdani
Russell Greiner
Yoshua Bengio
2021-11-07
Canadian Medical Association Journal (published)
doi.org
Problems in the deployment of machine-learned models in health care
Joseph Paul Cohen
Tianshi Cao
Joseph D Viviano
Chinwei Huang
Michael Fralick
Marzyeh Ghassemi
M. Mamdani
R. Greiner
Yoshua Bengio
2021-08-30
Canadian Medical Association Journal (published)
doi.org
Convex Potential Flows: Universal Probability Distributions with Optimal Transport and Convex Optimization
Chin-wei Huang
Ricky T. Q. Chen
Christos Tsirigotis
Aaron Courville
Flow-based models are powerful tools for designing probabilistic models with tractable density. This paper introduces Convex Potential Flows… (see more) (CP-Flow), a natural and efficient parameterization of invertible models inspired by the optimal transport (OT) theory. CP-Flows are the gradient map of a strongly convex neural potential function. The convexity implies invertibility and allows us to resort to convex optimization to solve the convex conjugate for efficient inversion. To enable maximum likelihood training, we derive a new gradient estimator of the log-determinant of the Jacobian, which involves solving an inverse-Hessian vector product using the conjugate gradient method. The gradient estimator has constant-memory cost, and can be made effectively unbiased by reducing the error tolerance level of the convex optimization routine. Theoretically, we prove that CP-Flows are universal density approximators and are optimal in the OT sense. Our empirical results show that CP-Flow performs competitively on standard benchmarks of density estimation and variational inference.
2021-01-01
ICLR (published)
openreview.net
openreview.net
A Variational Perspective on Diffusion-Based Generative Models and Score Matching
Chin-wei Huang
Jae Hyun Lim
Aaron Courville
Discrete-time diffusion-based generative models and score matching methods have shown promising results in modeling high-dimensional image d… (see more)ata. Recently, Song et al. (2021) show that diffusion processes that transform data into noise can be reversed via learning the score function, i.e. the gradient of the log-density of the perturbed data. They propose to plug the learned score function into an inverse formula to define a generative diffusion process. Despite the empirical success, a theoretical underpinning of this procedure is still lacking. In this work, we approach the (continuous-time) generative diffusion directly and derive a variational framework for likelihood estimation, which includes continuous-time normalizing flows as a special case, and can be seen as an infinitely deep variational autoencoder. Under this framework, we show that minimizing the score-matching loss is equivalent to maximizing a lower bound of the likelihood of the plug-in reverse SDE proposed by Song et al. (2021), bridging the theoretical gap.
openreview.net
Bijective-Contrastive Estimation
Jae Hyun Lim
Chin-wei Huang
Aaron Courville
Chris Pal
In this work, we propose Bijective-Contrastive Estimation (BCE), a classification-based learning criterion for energy-based models. We gener… (see more)ate a collection of contrasting distributions using bijections, and solve all the classification problems between the original data distribution and the distributions induced by the bijections using a classifier parameterized by an energy model. We show that if the classification objective is minimized, the energy function will uniquely recover the data density up to a normalizing constant. This has the benefit of not having to explicitly specify a contrasting distribution, like noise contrastive estimation. Experimentally, we demonstrate that the proposed method works well on 2D synthetic datasets. We discuss the difficulty in high dimensional cases, and propose potential directions to explore for future work.
2020-12-21
approximateinference.org/AABI/2021/Symposium (accepted)
openreview.net
openreview.net
AR-DAE: Towards Unbiased Neural Entropy Gradient Estimation
Jae Hyun Lim
Aaron Courville
Chris Pal
Chin-wei Huang
2020-11-21
Proceedings of the 37th International Conference on Machine Learning (published)
proceedings.mlr.press
arxiv.org
Stochastic Neural Network with Kronecker Flow
Chin-wei Huang
Ahmed Touati
Pascal Vincent
Gintare Karolina Dziugaite
Alexandre Lacoste
Aaron Courville
Recent advances in variational inference enable the modelling of highly structured joint distributions, but are limited in their capacity to… (see more) scale to the high-dimensional setting of stochastic neural networks. This limitation motivates a need for scalable parameterizations of the noise generation process, in a manner that adequately captures the dependencies among the various parameters. In this work, we address this need and present the Kronecker Flow, a generalization of the Kronecker product to invertible mappings designed for stochastic neural networks. We apply our method to variational Bayesian neural networks on predictive tasks, PAC-Bayes generalization bound estimation, and approximate Thompson sampling in contextual bandits. In all setups, our methods prove to be competitive with existing methods and better than the baselines.
2020-06-03
Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics (published)
proceedings.mlr.press
arxiv.org
Solving ODE with Universal Flows: Approximation Theory for Flow-Based Models
Chin-wei Huang
Laurent Dinh
Aaron Courville
Normalizing flows are powerful invertible probabilistic models that can be used to translate two probability distributions, in a way that al… (see more)lows us to efficiently track the change of probability density. However, to trade for computational efficiency in sampling and in evaluating the log-density, special parameterization designs have been proposed at the cost of representational expressiveness. In this work, we propose to use ODEs as a framework to establish universal approximation theory for certain families of flow-based models.
2020-02-26
International Conference on Learning Representations (published)
openreview.net
openreview.net
Augmented Normalizing Flows: Bridging the Gap Between Generative Flows and Latent Variable Models
Chin-wei Huang
Laurent Dinh
Aaron Courville
In this work, we propose a new family of generative flows on an augmented data space, with an aim to improve expressivity without drasticall… (see more)y increasing the computational cost of sampling and evaluation of a lower bound on the likelihood. Theoretically, we prove the proposed flow can approximate a Hamiltonian ODE as a universal transport map. Empirically, we demonstrate state-of-the-art performance on standard benchmarks of flow-based generative modeling.
2020-02-17
ArXiv (preprint)
arxiv.org
arxiv.org
Investigating Biases in Textual Entailment Datasets
Shawn Tan
Yikang Shen
Chin-wei Huang
Aaron Courville
The ability to understand logical relationships between sentences is an important task in language understanding. To aid in progress for thi… (see more)s task, researchers have collected datasets for machine learning and evaluation of current systems. However, like in the crowdsourced Visual Question Answering (VQA) task, some biases in the data inevitably occur. In our experiments, we find that performing classification on just the hypotheses on the SNLI dataset yields an accuracy of 64%. We analyze the bias extent in the SNLI and the MultiNLI dataset, discuss its implication, and propose a simple method to reduce the biases in the datasets.
2019-06-23
ArXiv (preprint)
arxiv.org
arxiv.org