Portrait of Frank Wood is unavailable

Frank Wood

Affiliate Member
Associate Professor, University of British Columbia, Department of Computer Science
CEO, Inverted AI
Research Topics
Deep Learning
Generative Models
Probabilistic Models
Reinforcement Learning

Biography

Frank Wood is an associate professor of computer science at the University of British Columbia, and an Affiliate member at Mila – Quebec Artificial Intelligence Institute. He is also the CEO of Inverted AI.

His research interests include probabilistic programming, as well as automatic learning and probabilistic AI. He is particularly interested in Bayesian methods and unsupervised learning.

Publications

Planning as Inference in Epidemiological Models
Frank Wood
Andrew Warrington
Saeid Naderiparizi
Christian Dietrich Weilbach
Vaden Masrani
William Harvey
Adam Ścibior
Boyan Beronov
Seyed Ali Nasseri
In this work we demonstrate how existing software tools can be used to automate parts of infectious disease-control policy-making via perfor… (see more)ming inference in existing epidemiological dynamics models. The kind of inference tasks undertaken include computing, for planning purposes, the posterior distribution over putatively controllable, via direct policy-making choices, simulation model parameters that give rise to acceptable disease progression outcomes. Neither the full capabilities of such inference automation software tools nor their utility for planning is widely disseminated at the current time. Timely gains in understanding about these tools and how they can be used may lead to more fine-grained and less economically damaging policy prescriptions, particularly during the current COVID-19 pandemic.
2020-03-30
arXiv.org (preprint)
dblp.uni-trier.de
Coping With Simulators That Don't Always Return
Andrew Warrington
Saeid Naderiparizi
Frank Wood
Deterministic models are approximations of reality that are easy to interpret and often easier to build than stochastic alternatives. Unfort… (see more)unately, as nature is capricious, observational data can never be fully explained by deterministic models in practice. Observation and process noise need to be added to adapt deterministic models to behave stochastically, such that they are capable of explaining and extrapolating from noisy data. We investigate and address computational inefficiencies that arise from adding process noise to deterministic simulators that fail to return for certain inputs; a property we describe as "brittle." We show how to train a conditional normalizing flow to propose perturbations such that the simulator succeeds with high probability, increasing computational efficiency.
2020-03-28
ArXiv (preprint)
arxiv.org
arxiv.org
Coping With Simulators That Don't Always Return
Andrew Warrington
Saeid Naderiparizi
Frank Wood
2020-01-01
AISTATS (published)
proceedings.mlr.press
arxiv.org
Structured Conditional Continuous Normalizing Flows for Efficient Amortized Inference in Graphical Models
Christian Dietrich Weilbach
Boyan Beronov
Frank Wood
William Harvey
We exploit minimally faithful inversion of graphical model structures to specify sparse continuous normalizing flows (CNFs) for amortized i… (see more)nference. We find that the sparsity of this factorization can be exploited to reduce the numbers of parameters in the neural network, adaptive integration steps of the flow, and consequently FLOPs at both training and inference time without decreasing performance in comparison to unconstrained flows. By expressing the structure inversion as a compilation pass in a probabilistic programming language, we are able to apply it in a novel way to models as complex as convolutional neural networks. Furthermore, we extend the training objective for CNFs in the context of inference amortization to the symmetric Kullback-Leibler divergence, and demonstrate its theoretical and practical advantages.
2020-01-01
International Conference on Artificial Intelligence and Statistics (published)
dblp.uni-trier.de
Structured Conditional Continuous Normalizing Flows for Efficient Amortized Inference in Graphical Models
Christian Dietrich Weilbach
Boyan Beronov
Frank Wood
William Harvey
We exploit minimally faithful inversion of graphical model structures to specify sparse continuous normalizing flows (CNFs) for amortized i… (see more)nference. We find that the sparsity of this factorization can be exploited to reduce the numbers of parameters in the neural network, adaptive integration steps of the flow, and consequently FLOPs at both training and inference time without decreasing performance in comparison to unconstrained flows. By expressing the structure inversion as a compilation pass in a probabilistic programming language, we are able to apply it in a novel way to models as complex as convolutional neural networks. Furthermore, we extend the training objective for CNFs in the context of inference amortization to the symmetric Kullback-Leibler divergence, and demonstrate its theoretical and practical advantages.
2020-01-01
International Conference on Artificial Intelligence and Statistics (published)
proceedings.mlr.press
Near-Optimal Glimpse Sequences for Improved Hard Attention Neural Network Training
William Harvey
Michael Teng
Frank Wood
Hard visual attention is a promising approach to reduce the computational burden of modern computer vision methodologies. However, hard atte… (see more)ntion mechanisms can be difficult and slow to train, which is especially costly for applications like neural architecture search where multiple networks must be trained. We introduce a method to amortise the cost of training by generating an extra supervision signal for a subset of the training data. This supervision is in the form of sequences of ‘good’ locations to attend to for each image. We find that the best method to generate supervision sequences comes from framing hard attention for image classification as a Bayesian optimal experimental design (BOED) problem. From this perspective, the optimal locations to attend to are those which provide the greatest expected reduction in the entropy of the classification distribution. We introduce methodology from the BOED literature to approximate this optimal behaviour and generate ‘near-optimal’ supervision sequences. We then present a hard attention network training objective that makes use of these sequences and show that it allows faster training than prior work. We finally demonstrate the utility of faster hard attention training by incorporating supervision sequences in a neural architecture search, resulting in hard attention architectures which can outperform networks with access to the entire image.
2019-06-13
ArXiv (preprint)
doi.org
arxiv.org
LF-PPL: A Low-Level First Order Probabilistic Programming Language for Non-Differentiable Models
Yuanshuo Zhou
Bradley Gram-Hansen
Tobias Kohn
Tom Rainforth
Hongseok Yang
Frank Wood
We develop a new Low-level, First-order Probabilistic Programming Language~(LF-PPL) suited for models containing a mix of continuous, discre… (see more)te, and/or piecewise-continuous variables. The key success of this language and its compilation scheme is in its ability to automatically distinguish parameters the density function is discontinuous with respect to, while further providing runtime checks for boundary crossings. This enables the introduction of new inference engines that are able to exploit gradient information, while remaining efficient for models which are not everywhere differentiable. We demonstrate this ability by incorporating a discontinuous Hamiltonian Monte Carlo (DHMC) inference engine that is able to deliver automated and efficient inference for non-differentiable models. Our system is backed up by a mathematical formalism that ensures that any model expressed in this language has a density with measure zero discontinuities to maintain the validity of the inference engine.
2019-03-06
ArXiv (preprint)
arxiv.org
arxiv.org
LF-PPL: A Low-Level First Order Probabilistic Programming Language for Non-Differentiable Models
Yuanshuo Zhou
Bradley Gram-Hansen
Tobias Kohn
Tom Rainforth
Hongseok Yang
Frank Wood
We develop a new Low-level, First-order Probabilistic Programming Language~(LF-PPL) suited for models containing a mix of continuous, discre… (see more)te, and/or piecewise-continuous variables. The key success of this language and its compilation scheme is in its ability to automatically distinguish parameters the density function is discontinuous with respect to, while further providing runtime checks for boundary crossings. This enables the introduction of new inference engines that are able to exploit gradient information, while remaining efficient for models which are not everywhere differentiable. We demonstrate this ability by incorporating a discontinuous Hamiltonian Monte Carlo (DHMC) inference engine that is able to deliver automated and efficient inference for non-differentiable models. Our system is backed up by a mathematical formalism that ensures that any model expressed in this language has a density with measure zero discontinuities to maintain the validity of the inference engine.
2019-01-01
AISTATS (published)
proceedings.mlr.press
arxiv.org