Derek Nowrouzezahrai

Carsten Dachsbacher

The design of the proposal distributions, and most notably the kernel parameters, are crucial for the performance of Markov chain Monte Carl… (see more)o (MCMC) rendering. A poor selection of parameters can increase the correlation of the Markov chain and result in bad rendering performance. We approach this problem by a novel path perturbation strategy for online-learning of state-dependent kernel parameters. We base our approach on the theoretical framework of regional adaptive MCMC which enables the adaptation of parameters depending on the region of the state space which contains the current sample, and on information collected from previous samples. For this, we define a partitioning of the path space on a low-dimensional canonical space to capture the characteristics of paths, with a focus on path segments closer to the sensor. Fast convergence is achieved by adaptive refinement of the partitions. Exemplarily, we present two novel regional adaptive path perturbation techniques akin to lens and multi-chain perturbations. Our approach can easily be used on top of existing path space MLT methods to improve rendering efficiency, while being agnostic to the initial choice of kernel parameters.

2024-02-13

ArXiv (preprint)

Minimax Exploiter: A Data Efficient Approach for Competitive Self-Play

Daniel Bairamian

Philippe Marcotte

Joshua Romoff

Gabriel Robert

2024-01-01

AAMAS (published)

Cone-Traced Supersampling with Subpixel Edge Reconstruction.

Andrei Chubarau

Yangyang Zhao

Ruby Rao

Paul Kry

While signed distance fields (SDFs) in theory offer infinite level of detail, they are typically rendered using the sphere tracing algorithm… (see more) at finite resolutions, which causes the common rasterized image synthesis problem of aliasing. Most existing optimized antialiasing solutions rely on polygon mesh representations; SDF-based geometry can only be directly antialiased with the computationally expensive supersampling or with post-processing filters that may produce undesirable blurriness and ghosting. In this work, we present cone-traced supersampling (CTSS), an efficient and robust spatial antialiasing solution that naturally complements the sphere tracing algorithm, does not require casting additional rays per pixel or offline prefiltering, and can be easily implemented in existing real-time SDF renderers. CTSS performs supersampling along the traced ray near surfaces with partial visibility – object contours – identified by evaluating cone intersections within a pixel's view frustum. We further introduce subpixel edge reconstruction (SER), a technique that extends CTSS to locate and resolve complex pixels with geometric edges in relatively flat regions, which are otherwise undetected by cone intersections. Our combined solution relies on a specialized sampling strategy to minimize the number of shading computations and correlates sample visibility to aggregate the samples. With comparable antialiasing quality at significantly lower computational cost, CTSS is a reliable practical alternative to conventional supersampling.

2023-12-14

IEEE Transactions on Visualization and Computer Graphics (published)

Efficient Graphics Representation with Differentiable Indirection

Sayantan Datta

Carl Marshall

Zhao Dong

Zhengqin Li

We introduce differentiable indirection – a novel learned primitive that employs differentiable multi-scale lookup tables as an effective … (see more)substitute for traditional compute and data operations across the graphics pipeline. We demonstrate its flexibility on a number of graphics tasks, i.e., geometric and image representation, texture mapping, shading, and radiance field representation. In all cases, differentiable indirection seamlessly integrates into existing architectures, trains rapidly, and yields both versatile and efficient results.

2023-12-11

SIGGRAPH Asia 2023 Conference Papers (published)

Differentiable visual computing for inverse problems and machine learning

Andrew Spielberg

Fangcheng Zhong

Konstantinos Rematas

Krishna Murthy

Cengiz Oztireli

Tzu-Mao Li

2023-11-17

Nature Machine Intelligence (published)

STAMP: Differentiable Task and Motion Planning via Stein Variational Gradient Descent

Yewon Lee

Philip Huang

Yizhou Huang

Krishna Murthy

Andrew Zou Li

Fabian Damken

Eric Heiden

Kevin A. Smith

Fabio Ramos

Florian Shkurti

Carnegie-mellon University

M. I. O. Technology

Technische Universitat Darmstadt

Nvidia

McGill University

University of Sydney

Planning for many manipulation tasks, such as using tools or assembling parts, often requires both symbolic and geometric reasoning. Task an… (see more)d Motion Planning (TAMP) algorithms typically solve these problems by conducting a tree search over high-level task sequences while checking for kinematic and dynamic feasibility. While performant, most existing algorithms are highly inefficient as their time complexity grows exponentially with the number of possible actions and objects. Additionally, they only find a single solution to problems in which many feasible plans may exist. To address these limitations, we propose a novel algorithm called Stein Task and Motion Planning (STAMP) that leverages parallelization and differentiable simulation to efficiently search for multiple diverse plans. STAMP relaxes discrete-and-continuous TAMP problems into continuous optimization problems that can be solved using variational inference. Our algorithm builds upon Stein Variational Gradient Descent, a gradient-based variational inference algorithm, and parallelized differentiable physics simulators on the GPU to efficiently obtain gradients for inference. Further, we employ imitation learning to introduce action abstractions that reduce the inference problem to lower dimensions. We demonstrate our method on two TAMP problems and empirically show that STAMP is able to: 1) produce multiple diverse plans in parallel; and 2) search for plans more efficiently compared to existing TAMP baselines.

2023-10-23

robot-learning.org/CoRL/2023/Workshop/LEAP (poster)

openreview.net

Learning Neural Implicit Representations with Surface Signal Parameterizations

Yanran Guan

Andrei Chubarau

Ruby Rao

2023-08-01

Computers & Graphics (published)

Parameter-space ReSTIR for Differentiable and Inverse Rendering

Wesley Chang

Venkataram Sivaram

Toshiya Hachisuka

Ravi Ramamoorthi

Tzu-Mao Li

Differentiable rendering is frequently used in gradient descent-based inverse rendering pipelines to solve for scene parameters – such as … (see more)reflectance or lighting properties – from target image inputs. Efficient computation of accurate, low variance gradients is critical for rapid convergence. While many methods employ variance reduction strategies, they operate independently on each gradient descent iteration, requiring large sample counts and computation. Gradients may however vary slowly between iterations, leading to unexplored potential benefits when reusing sample information to exploit this coherence. We develop an algorithm to reuse Monte Carlo gradient samples between gradient iterations, motivated by reservoir-based temporal importance resampling in forward rendering. Direct application of this method is not feasible, as we are computing many derivative estimates (i.e., one per optimization parameter) instead of a single pixel intensity estimate; moreover, each of these gradient estimates can affect multiple pixels, and gradients can take on negative values. We address these challenges by reformulating differential rendering integrals in parameter space, developing a new resampling estimator that treats negative functions, and combining these ideas into a reuse algorithm for inverse texture optimization. We significantly reduce gradient error compared to baselines, and demonstrate faster inverse rendering convergence in settings involving complex direct lighting and material textures.

2023-07-23

Special Interest Group on Computer Graphics and Interactive Techniques Conference Conference Proceedings (published)

A Model-Based Solution to the Offline Multi-Agent Reinforcement Learning Coordination Problem

Paul Barde

Jakob Nicolaus Foerster

Amy Zhang

2023-05-26

ArXiv (preprint)

Cone-Traced Supersampling for Signed Distance Field Rendering

Andrei Chubarau

Yangyang Zhao

Ruby Rao

Paul Kry

While Signed Distance Fields (SDFs) in theory offer infinite level of detail, they are typically rendered using the sphere tracing algorithm… (see more) at finite resolutions, which causes the common rasterized image synthesis problem of aliasing. Most existing optimized antialiasing solutions rely on polygon mesh representations; SDF-based geometry can only be directly antialiased with the computationally expensive supersampling or with post-processing filters that often lead to undesirable blurriness and ghosting. In this work, we present cone-traced supersampling (CTSS), an efficient and robust spatial antialiasing solution that naturally complements the sphere tracing algorithm, does not require casting additional rays per pixel or offline pre-filtering, and can be easily implemented in existing real-time SDF renderers. CTSS performs supersampling along the traced ray near surfaces with partial visibility identified by evaluating cone intersections within a pixel's view frustum. We further devise a specialized sampling strategy to minimize the number of shading computations and aggregate the collected samples based on their correlated visibility. Depending on configuration, CTSS incurs roughly 15-30% added computational cost and significantly outperforms conventional supersampling approaches while offering comparative antialiasing and visual image quality for most geometric edges.

2023-05-02

graphicsinterface.org/Graphics_Interface/2023/Conference_SD (published)

openreview.net

Visual Question Answering From Another Perspective: CLEVR Mental Rotation Tests

Christopher Beckham

Martin Weiss

Florian Golemo

Sina Honari

Chris Pal

2023-04-01

Pattern Recognition (published)

MeshDiffusion: Score-based Generative 3D Mesh Modeling

Zhen Liu

Yao Feng

Michael J. Black

Liam Paull

Weiyang Liu

We consider the task of generating realistic 3D shapes, which is useful for a variety of applications such as automatic scene generation and… (see more) physical simulation. Compared to other 3D representations like voxels and point clouds, meshes are more desirable in practice, because (1) they enable easy and arbitrary manipulation of shapes for relighting and simulation, and (2) they can fully leverage the power of modern graphics pipelines which are mostly optimized for meshes. Previous scalable methods for generating meshes typically rely on sub-optimal post-processing, and they tend to produce overly-smooth or noisy surfaces without fine-grained geometric details. To overcome these shortcomings, we take advantage of the graph structure of meshes and use a simple yet very effective generative modeling method to generate 3D meshes. Specifically, we represent meshes with deformable tetrahedral grids, and then train a diffusion model on this direct parameterization. We demonstrate the effectiveness of our model on multiple generative tasks.

2023-02-01

ICLR.cc/2023/Conference (notable)