Portrait de Vikram Voleti n'est pas disponible

Vikram Voleti

Alumni

Publications

Multi-resolution continuous normalizing flows.
Vikram Voleti
Chris Finlay
Adam Oberman
Christopher Pal
Recent work has shown that Neural Ordinary Differential Equations (ODEs) can serve as generative models of images using the perspective of C… (voir plus)ontinuous Normalizing Flows (CNFs). Such models offer exact likelihood calculation, and invertible generation/density estimation. In this work we introduce a Multi-Resolution variant of such models (MRCNF), by characterizing the conditional distribution over the additional information required to generate a fine image that is consistent with the coarse image. We introduce a transformation between resolutions that allows for no change in the log likelihood. We show that this approach yields comparable likelihood values for various image datasets, with improved performance at higher resolutions, with fewer parameters, using only one GPU. Further, we examine the out-of-distribution properties of MRCNFs, and find that they are similar to those of other likelihood-based generative models.
2024-03-20
Annals of Mathematics and Artificial Intelligence (inconnu)
doi.org
openreview.net
Are Diffusion Models Vision-And-Language Reasoners?
Benno Krojer
Elinor Poole-Dayan
Vikram Voleti
Christopher Pal
Siva Reddy
Text-conditioned image generation models have recently shown immense qualitative success using denoising diffusion processes. However, unlik… (voir plus)e discriminative vision-and-language models, it is a non-trivial task to subject these diffusion-based generative models to automatic fine-grained quantitative evaluation of high-level phenomena such as compositionality. Towards this goal, we perform two innovations. First, we transform diffusion-based models (in our case, Stable Diffusion) for any image-text matching (ITM) task using a novel method called DiffusionITM. Second, we introduce the Generative-Discriminative Evaluation Benchmark (GDBench) benchmark with 7 complex vision-and-language tasks, bias evaluation and detailed analysis. We find that Stable Diffusion + DiffusionITM is competitive on many tasks and outperforms CLIP on compositional tasks like like CLEVR and Winoground. We further boost its compositional performance with a transfer setup by fine-tuning on MS-COCO while retaining generative capabilities. We also measure the stereotypical bias in diffusion models, and find that Stable Diffusion 2.1 is, for the most part, less biased than Stable Diffusion 1.5. Overall, our results point in an exciting direction bringing discriminative and generative model evaluation closer. We will release code and benchmark setup soon.
2023-09-20
NeurIPS.cc/2023/Conference (poster)
doi.org
openreview.net
Score-based Diffusion Models in Function Space
Jae Hyun Lim
Nikola B. Kovachki
R. Baptista
Christopher Beckham
Kamyar Azizzadenesheli
Jean Kossaifi
Vikram Voleti
Jiaming Song
Karsten Kreis
Jan Kautz
Christopher Pal
Arash Vahdat
Animashree Anandkumar
2023-02-13
ArXiv (prépublication)
doi.org
arxiv.org
Score-based Denoising Diffusion with Non-Isotropic Gaussian Noise Models
Vikram Voleti
Christopher Pal
Adam Oberman
Generative models based on denoising diffusion techniques have led to an unprecedented increase in the quality and diversity of imagery that… (voir plus) is now possible to create with neural generative models. However, most contemporary state-of-the-art methods are derived from a standard isotropic Gaussian formulation. In this work we examine the situation where non-isotropic Gaussian distributions are used. We present the key mathematical derivations for creating denoising diffusion models using an underlying non-isotropic Gaussian noise model. We also provide initial experiments with the CIFAR10 dataset to help verify empirically that this more general modelling approach can also yield high-quality samples.
2022-11-28
NeurIPS.cc/2022/Workshop/SBM (poster)
doi.org
openreview.net
SMPL-IK: Learned Morphology-Aware Inverse Kinematics for AI Driven Artistic Workflows
Vikram Voleti
Boris N. Oreshkin
Florent Bocquelet
Félix G. Harvey
Louis-Simon Ménard
Christopher Pal
Inverse Kinematics (IK) systems are often rigid with respect to their input character, thus requiring user intervention to be adapted to new… (voir plus) skeletons. In this paper we aim at creating a flexible, learned IK solver applicable to a wide variety of human morphologies. We extend a state-of-the-art machine learning IK solver to operate on the well known Skinned Multi-Person Linear model (SMPL). We call our model SMPL-IK, and show that when integrated into real-time 3D software, this extended system opens up opportunities for defining novel AI-assisted animation workflows. For example, pose authoring can be made more flexible with SMPL-IK by allowing users to modify gender and body shape while posing a character. Additionally, when chained with existing pose estimation algorithms, SMPL-IK accelerates posing by allowing users to bootstrap 3D scenes from 2D images while allowing for further editing. Finally, we propose a novel SMPL Shape Inversion mechanism (SMPL-SI) to map arbitrary humanoid characters to the SMPL space, allowing artists to leverage SMPL-IK on custom characters. In addition to qualitative demos showing proposed tools, we present quantitative SMPL-IK baselines on the H36M and AMASS datasets.
2022-11-21
SIGGRAPH Asia 2022 Technical Communications (publié)
doi.org
arxiv.org
Generative Models of Brain Dynamics
Mahta Ramezanian-Panahi
Germán Abrevaya
Jean-christophe Gagnon-audet
Vikram Voleti
Irina Rish
Guillaume Dumas
2022-07-14
Frontiers in Artificial Intelligence (publié)
doi.org
MCVD: Masked Conditional Video Diffusion for Prediction, Generation, and Interpolation
Vikram Voleti
Alexia Jolicoeur-Martineau
Christopher Pal
2021-12-31
Advances in Neural Information Processing Systems 35 (NeurIPS 2022) (publié)
doi.org
openreview.net
Generative Models of Brain Dynamics -- A review
Mahta Ramezanian Panahi
Germán Abrevaya
Jean-christophe Gagnon-audet
Vikram Voleti
Irina Rish
Guillaume Dumas
The principled design and discovery of biologically- and physically-informed models of neuronal dynamics has been advancing since the mid-tw… (voir plus)entieth century. Recent developments in artificial intelligence (AI) have accelerated this progress. This review article gives a high-level overview of the approaches across different scales of organization and levels of abstraction. The studies covered in this paper include fundamental models in computational neuroscience, nonlinear dynamics, data-driven methods, as well as emergent practices. While not all of these models span the intersection of neuroscience, AI, and system dynamics, all of them do or can work in tandem as generative models, which, as we argue, provide superior properties for the analysis of neuroscientific data. We discuss the limitations and unique dynamical traits of brain data and the complementary need for hypothesis- and data-driven modeling. By way of conclusion, we present several hybrid generative models from recent literature in scientific machine learning, which can be efficiently deployed to yield interpretable models of neural dynamics.
2021-12-21
ArXiv (prépublication)
arxiv.org
arxiv.org
Simple Video Generation using Neural ODEs
David Kanaa
Vikram Voleti
S Ebrahimi Kahou
Christopher Pal
Despite having been studied to a great extent, the task of conditional generation of sequences of frames, or videos, remains extremely chall… (voir plus)enging. It is a common belief that a key step towards solving this task resides in modelling accurately both spatial and temporal information in video signals. A promising direction to do so has been to learn latent variable models that predict the future in latent space and project back to pixels, as suggested in recent literature. Following this line of work and building on top of a family of models introduced in prior work, Neural ODE, we investigate an approach that models time-continuous dynamics over a continuous latent space with a differential equation with respect to time. The intuition behind this approach is that these trajectories in latent space could then be extrapolated to generate video frames beyond the time steps for which the model is trained. We show that our approach yields promising results in the task of future frame prediction on the Moving MNIST dataset with 1 and 2 digits.
2021-09-06
ArXiv (prépublication)
arxiv.org
arxiv.org
Improving Continuous Normalizing Flows using a Multi-Resolution Framework
Vikram Voleti
Chris Finlay
Adam Oberman
Christopher Pal
Recent work has shown that Continuous Normalizing Flows (CNFs) can serve as generative models of images with exact likelihood calculation an… (voir plus)d invertible generation/density estimation. In this work we introduce a Multi-Resolution variant of such models (MRCNF). We introduce a transformation between resolutions that allows for no change in the log likelihood. We show that this approach yields comparable likelihood values for various image datasets, with improved performance at higher resolutions, with fewer parameters, using only 1 GPU.
2021-06-14
ICML.cc/2021/Workshop/INNF (poster)
openreview.net
openreview.net
Accounting for Variance in Machine Learning Benchmarks
Xavier Bouthillier
Pierre Delaunay
Mirko Bronzi
Assya Trofimov
Brennan Nichyporuk
Justin Szeto
Naz Sepah
Edward Raff
Kanika Madan
Vikram Voleti
Samira Ebrahimi Kahou
Vincent Michalski
Dmitriy Serdyuk
Tal Arbel
Chris Pal
Gael Varoquaux
Pascal Vincent
Strong empirical evidence that one machine-learning algorithm A outperforms another one B ideally calls for multiple trials optimizing the l… (voir plus)earning pipeline over sources of variation such as data sampling, data augmentation, parameter initialization, and hyperparameters choices. This is prohibitively expensive, and corners are cut to reach conclusions. We model the whole benchmarking process, revealing that variance due to data sampling, parameter initialization and hyperparameter choice impact markedly the results. We analyze the predominant comparison methods used today in the light of this variance. We show a counter-intuitive result that adding more sources of variation to an imperfect estimator approaches better the ideal estimator at a 51 times reduction in compute cost. Building on these results, we study the error rate of detecting improvements, on five different deep-learning tasks/architectures. This study leads us to propose recommendations for performance comparisons.
2020-12-31
MLSys (publié)
doi.org
arxiv.org
Learning to Combine Top-Down and Bottom-Up Signals in Recurrent Neural Networks with Attention over Modules
Sarthak Mittal
Alex Lamb
Anirudh Goyal
Vikram Voleti
Murray Shanahan
Guillaume Lajoie
Michael Mozer
Yoshua Bengio
Robust perception relies on both bottom-up and top-down signals. Bottom-up signals consist of what's directly observed through sensation. To… (voir plus)p-down signals consist of beliefs and expectations based on past experience and short-term memory, such as how the phrase `peanut butter and~...' will be completed. The optimal combination of bottom-up and top-down information remains an open question, but the manner of combination must be dynamic and both context and task dependent. To effectively utilize the wealth of potential top-down information available, and to prevent the cacophony of intermixed signals in a bidirectional architecture, mechanisms are needed to restrict information flow. We explore deep recurrent neural net architectures in which bottom-up and top-down signals are dynamically combined using attention. Modularity of the architecture further restricts the sharing and communication of information. Together, attention and modularity direct information flow, which leads to reliable performance improvements in perceptual and language tasks, and in particular improves robustness to distractions and noisy data. We demonstrate on a variety of benchmarks in language modeling, sequential image classification, video prediction and reinforcement learning that the \emph{bidirectional} information flow can improve results over strong baselines.
2019-12-31
ICML (publié)
doi.org
proceedings.mlr.press