Christopher Beckham

Deep Learning for Detecting Extreme Weather Patterns

Mayur Mudigonda

Mayur Mudigonda, Prabhat Ram

Prabhat Ram

Karthik Kashinath

Evan Racah

Ankur Mahesh

Yunjie Liu

Christopher Beckham

Jim Biard

Thorsten Kurth

Sookyung Kim

Samira Ebrahimi Kahou

Tegan Maharaj

Burlen Loring

Chris Pal

Travis O'Brien

K. Kunkel

Kenneth E. Kunkel

M. Wehner

Michael F. Wehner … (see 2 more)

W. Collins

William D. Collins

2021-08-20

Deep Learning for the Earth Sciences (published)

doi.org

Manifold Mixup: Better Representations by Interpolating Hidden States

Amir Najafi

David Lopez-Paz

Deep neural networks excel at learning the training data, but often provide incorrect and confident predictions when evaluated on slightly d… (see more)ifferent test examples. This includes distribution shifts, outliers, and adversarial examples. To address these issues, we propose Manifold Mixup, a simple regularizer that encourages neural networks to predict less confidently on interpolations of hidden representations. Manifold Mixup leverages semantic interpolations as additional training signal, obtaining neural networks with smoother decision boundaries at multiple levels of representation. As a result, neural networks trained with Manifold Mixup learn class-representations with fewer directions of variance. We prove theory on why this flattening happens under ideal conditions, validate it on practical situations, and connect it to previous works on information theory and generalization. In spite of incurring no significant computation and being implemented in a few lines of code, Manifold Mixup improves strong baselines in supervised learning, robustness to single-step adversarial attacks, and test log-likelihood.

2019-05-24

Proceedings of the 36th International Conference on Machine Learning (published)

proceedings.mlr.press

On Adversarial Mixup Resynthesis

In this paper, we explore new approaches to combining information encoded within the learned representations of auto-encoders. We explore mo… (see more)dels that are capable of combining the attributes of multiple inputs such that a resynthesised output is trained to fool an adversarial discriminator for real versus synthesised data. Furthermore, we explore the use of such an architecture in the context of semi-supervised learning, where we learn a mixing function whose objective is to produce interpolations of hidden states, or masked combinations of latent representations that are consistent with a conditioned class label. We show quantitative and qualitative evidence that such a formulation is an interesting avenue of research.

Adversarial Mixup Resynthesizers

In this paper, we explore new approaches to combining information encoded within the learned representations of autoencoders. We explore mod… (see more)els that are capable of combining the attributes of multiple inputs such that a resynthesised output is trained to fool an adversarial discriminator for real versus synthesised data. Furthermore, we explore the use of such an architecture in the context of semi-supervised learning, where we learn a mixing function whose objective is to produce interpolations of hidden states, or masked combinations of latent representations that are consistent with a conditioned class label. We show quantitative and qualitative evidence that such a formulation is an interesting avenue of research.

2019-01-01

DGS@ICLR (published)

openreview.net

Deep Learning recognizes weather and climate patterns

Karthik Kashinath

M. Prabhat

Mayur Mudigonda

Ankur Mahesh

Sookyung Kim

Yunjie Liu

Samira Ebrahimi Kahou

B. Toms

Evan Racah

Christopher Beckham

Chris Pal

Tegan Maharaj

Jim Biard

K. Kunkel

Dean Nesbit Williams

Travis O'Brien

M. Wehner

W. Collins

Manifold Mixup: Encouraging Meaningful On-Manifold Interpolation as a Regularizer

Deep networks often perform well on the data manifold on which they are trained, yet give incorrect (and often very confident) answers when … (see more)evaluated on points from off of the training distribution. This is exemplified by the adversarial examples phenomenon but can also be seen in terms of model generalization and domain shift. We propose Manifold Mixup which encourages the network to produce more reasonable and less confident predictions at points with combinations of attributes not seen in the training set. This is accomplished by training on convex combinations of the hidden state representations of data samples. Using this method, we demonstrate improved semi-supervised learning, learning with limited labeled data, and robustness to adversarial examples. Manifold Mixup requires no (significant) additional computation. Analytical experiments on both real data and synthetic data directly support our hypothesis for why the Manifold Mixup method improves results.

2018-06-13

arXiv.org (preprint)

dblp.uni-trier.de

Unsupervised Depth Estimation, 3D Face Rotation and Replacement

Joel Ruben Antony Moniz

We present an unsupervised approach for learning to estimate three dimensional (3D) facial structure from a single image while also predicti… (see more)ng 3D viewpoint transformations that match a desired pose and facial geometry. We achieve this by inferring the depth of facial keypoints of an input image in an unsupervised manner, without using any form of ground-truth depth information. We show how it is possible to use these depths as intermediate computations within a new backpropable loss to predict the parameters of a 3D affine transformation matrix that maps inferred 3D keypoints of an input face to the corresponding 2D keypoints on a desired target facial geometry or pose. Our resulting approach, called DepthNets, can therefore be used to infer plausible 3D transformations from one face pose to another, allowing faces to be frontalized, transformed into 3D models or even warped to another pose and facial geometry. Lastly, we identify certain shortcomings with our formulation, and explore adversarial image translation techniques as a post-processing step to re-synthesize complete head shots for faces re-targeted to different poses or identities.

arxiv.org

Unimodal Probability Distributions for Deep Ordinal Classification

Christopher Beckham

Chris Pal

Probability distributions produced by the cross-entropy loss for ordinal classification problems can possess undesired properties. We propos… (see more)e a straightforward technique to constrain discrete ordinal probability distributions to be unimodal via a combination of the Poisson probability mass function and the softmax nonlinearity. We evaluate this approach on two large ordinal image datasets and obtain promising results.

2017-07-17

Proceedings of the 34th International Conference on Machine Learning (published)

proceedings.mlr.press

arxiv.org

Custom AI Learning Programs

Mil'Haq Fest 2025

Mila Community of Practice

Supervision Requests

Christopher Beckham

Publications

Custom AI Learning Programs

Mil'Haq Fest 2025

Mila Community of Practice

Supervision Requests

Popular keywords:

Christopher Beckham

Publications