Frank Wood

Jakob H. Macke

2024-07-08

Proceedings of the 41st International Conference on Machine Learning (published)

Layerwise Proximal Replay: A Proximal Point Method for Online Continual Learning

Jinsoo Yoo

Yunpeng Liu

Geoff Pleiss

2024-07-08

Proceedings of the 41st International Conference on Machine Learning (published)

Nearest Neighbour Score Estimators for Diffusion Generative Models

Matthew Niedoba

Dylan Green

Saeid Naderiparizi

Vasileios Lioutas

Jonathan Wilder Lavington

Xiaoxuan Liang

Yunpeng Liu

Ke Zhang

Setareh Dabiri

Adam Ścibior

Berend Zwartsenberg

Score function estimation is the cornerstone of both training and sampling from diffusion generative models. Despite this fact, the most com… (see more)monly used estimators are either biased neural network approximations or high variance Monte Carlo estimators based on the conditional score. We introduce a novel nearest neighbour score function estimator which utilizes multiple samples from the training set to dramatically decrease estimator variance. We leverage our low variance estimator in two compelling applications. Training consistency models with our estimator, we report a significant increase in both convergence speed and sample quality. In diffusion models, we show that our estimator can replace a learned network for probability-flow ODE integration, opening promising new avenues of future research. Code will be released upon paper acceptance.

2024-07-08

Proceedings of the 41st International Conference on Machine Learning (published)

Prospective Messaging: Learning in Networks with Communication Delays

Ryan Fayyazi

Christian Dietrich Weilbach

Inter-neuron communication delays are ubiquitous in physically realized neural networks such as biological neural circuits and neuromorphic … (see more)hardware. These delays have significant and often disruptive consequences on network dynamics during training and inference. It is therefore essential that communication delays be accounted for, both in computational models of biological neural networks and in large-scale neuromorphic systems. Nonetheless, communication delays have yet to be comprehensively addressed in either domain. In this paper, we first show that delays prevent state-of-the-art continuous-time neural networks called Latent Equilibrium (LE) networks from learning even simple tasks despite significant overparameterization. We then propose to compensate for communication delays by predicting future signals based on currently available ones. This conceptually straightforward approach, which we call prospective messaging (PM), uses only neuron-local information, and is flexible in terms of memory and computation requirements. We demonstrate that incorporating PM into delayed LE networks prevents reaction lags, and facilitates successful learning on Fourier synthesis and autoregressive video prediction tasks.

2024-07-07

ArXiv (preprint)

Online Continual Learning of Video Diffusion Models From a Single Video Stream

Jinsoo Yoo

Dylan Green

Geoff Pleiss

2024-06-07

ArXiv (preprint)

TorchDriveEnv: A Reinforcement Learning Benchmark for Autonomous Driving with Reactive, Realistic, and Diverse Non-Playable Characters

Jonathan Wilder Lavington

Ke Zhang

Vasileios Lioutas

Matthew Niedoba

Yunpeng Liu

Dylan Green

Saeid Naderiparizi

Xiaoxuan Liang

Setareh Dabiri

Adam Ścibior

Berend Zwartsenberg

2024-05-07

ArXiv (preprint)

Semantically Consistent Video Inpainting with Conditional Diffusion Models

Dylan Green

William Harvey

Saeid Naderiparizi

Matthew Niedoba

Yunpeng Liu

Xiaoxuan Liang

Jonathan Wilder Lavington

Ke Zhang

Vasileios Lioutas

Setareh Dabiri

Adam Ścibior

Berend Zwartsenberg

Current state-of-the-art methods for video inpainting typically rely on optical flow or attention-based approaches to inpaint masked regions… (see more) by propagating visual information across frames. While such approaches have led to significant progress on standard benchmarks, they struggle with tasks that require the synthesis of novel content that is not present in other frames. In this paper we reframe video inpainting as a conditional generative modeling problem and present a framework for solving such problems with conditional video diffusion models. We highlight the advantages of using a generative approach for this task, showing that our method is capable of generating diverse, high-quality inpaintings and synthesizing new content that is spatially, temporally, and semantically consistent with the provided context.

2024-04-30

ArXiv (preprint)

On the Challenges and Opportunities in Generative AI

Laura Manduchi

Kushagra Pandey

Robert Bamler

Ryan Cotterell

Sina Daubener

Sophie Fellenz

Asja Fischer

Thomas Gartner

Matthias Kirchler

Marius Kloft

Yingzhen Li

Christoph Lippert

Gerard de Melo

Eric Nalisnick

Bjorn Ommer

Rajesh Ranganath

Maja Rudolph

Karen Ullrich

Guy Van den Broeck

Julia E Vogt … (see 5 more)

Yixin Wang

Florian Wenzel

Stephan Mandt

Vincent Fortuin

2024-02-28

ArXiv (preprint)

Nearest Neighbour Score Estimators for Diffusion Generative Models

Matthew Niedoba

Dylan Green

Saeid Naderiparizi

Vasileios Lioutas

Jonathan Wilder Lavington

Xiaoxuan Liang

Yunpeng Liu

Ke Zhang

Setareh Dabiri

Adam Ścibior

Berend Zwartsenberg

2024-02-12

ArXiv (preprint)

A Diffusion-Model of Joint Interactive Navigation

Matthew Niedoba

Jonathan Wilder Lavington

Yunpeng Liu

Vasileios Lioutas

Justice Sefas

Xiaoxuan Liang

Dylan Green

Setareh Dabiri

Berend Zwartsenberg

Adam Ścibior

Simulation of autonomous vehicle systems requires that simulated traffic participants exhibit diverse and realistic behaviors. The use of pr… (see more)erecorded real-world traffic scenarios in simulation ensures realism but the rarity of safety critical events makes large scale collection of driving scenarios expensive. In this paper, we present DJINN - a diffusion based method of generating traffic scenarios. Our approach jointly diffuses the trajectories of all agents, conditioned on a flexible set of state observations from the past, present, or future. On popular trajectory forecasting datasets, we report state of the art performance on joint trajectory metrics. In addition, we demonstrate how DJINN flexibly enables direct test-time sampling from a variety of valuable conditional distributions including goal-based sampling, behavior-class sampling, and scenario editing.

Don't be so negative! Score-based Generative Modeling with Oracle-assisted Guidance

Saeid Naderiparizi

Xiaoxuan Liang

Berend Zwartsenberg

2023-07-31

ArXiv (preprint)

Uncertain Evidence in Probabilistic Models and Stochastic Simulators

Andreas Munk

Alexander Mead

We consider the problem of performing Bayesian inference in probabilistic models where observations are accompanied by uncertainty, referred… (see more) to as "uncertain evidence.'' We explore how to interpret uncertain evidence, and by extension the importance of proper interpretation as it pertains to inference about latent variables. We consider a recently-proposed method "distributional evidence'' as well as revisit two older methods: Jeffrey's rule and virtual evidence. We devise guidelines on how to account for uncertain evidence and we provide new insights, particularly regarding consistency. To showcase the impact of different interpretations of the same uncertain evidence, we carry out experiments in which one interpretation is defined as "correct.'' We then compare inference results from each different interpretation illustrating the importance of careful consideration of uncertain evidence.

2023-07-03

Proceedings of the 40th International Conference on Machine Learning (published)