Derek Nowrouzezahrai

Fabio Ramos

Florian Shkurti

Carnegie-mellon University

M. I. O. Technology

Technische Universitat Darmstadt

Nvidia

M. University

University of Sydney

Planning for many manipulation tasks, such as using tools or assembling parts, often requires both symbolic and geometric reasoning. Task an… (see more)d Motion Planning (TAMP) algorithms typically solve these problems by conducting a tree search over high-level task sequences while checking for kinematic and dynamic feasibility. While performant, most existing algorithms are highly inefficient as their time complexity grows exponentially with the number of possible actions and objects. Additionally, they only find a single solution to problems in which many feasible plans may exist. To address these limitations, we propose a novel algorithm called Stein Task and Motion Planning (STAMP) that leverages parallelization and differentiable simulation to efficiently search for multiple diverse plans. STAMP relaxes discrete-and-continuous TAMP problems into continuous optimization problems that can be solved using variational inference. Our algorithm builds upon Stein Variational Gradient Descent, a gradient-based variational inference algorithm, and parallelized differentiable physics simulators on the GPU to efficiently obtain gradients for inference. Further, we employ imitation learning to introduce action abstractions that reduce the inference problem to lower dimensions. We demonstrate our method on two TAMP problems and empirically show that STAMP is able to: 1) produce multiple diverse plans in parallel; and 2) search for plans more efficiently compared to existing TAMP baselines.

2025-06-01

IEEE Robotics and Automation Letters (published)

openreview.net

Ctrl-Crash: Controllable Diffusion for Realistic Car Crashes

Anthony Gosselin

Ge Ya Luo

Luis Lara

Florian Golemo

Liam Paull

Alexia Jolicoeur-Martineau

Chris Pal

Video diffusion techniques have advanced significantly in recent years; however, they struggle to generate realistic imagery of car crashes … (see more)due to the scarcity of accident events in most driving datasets. Improving traffic safety requires realistic and controllable accident simulations. To tackle the problem, we propose Ctrl-Crash, a controllable car crash video generation model that conditions on signals such as bounding boxes, crash types, and an initial image frame. Our approach enables counterfactual scenario generation where minor variations in input can lead to dramatically different crash outcomes. To support fine-grained control at inference time, we leverage classifier-free guidance with independently tunable scales for each conditioning signal. Ctrl-Crash achieves state-of-the-art performance across quantitative video quality metrics (e.g., FVD and JEDi) and qualitative measurements based on a human-evaluation of physical realism and video quality compared to prior diffusion-based methods.

2025-05-30

ArXiv (preprint)

Ctrl-Crash: Controllable Diffusion for Realistic Car Crashes

Anthony Gosselin

Ge Ya Luo

Luis Lara

Florian Golemo

Liam Paull

Alexia Jolicoeur-Martineau

Chris Pal

2025-05-30

ArXiv (preprint)

Reinforcement Learning for Sequence Design Leveraging Protein Language Models

Jithendaraa Subramanian

Shiva Kanth Sujit

Niloy Irtisam

Umong Sain

Samira Ebrahimi Kahou

Riashat Islam

2024-07-03

ArXiv (preprint)

Regional Adaptive Metropolis Light Transport

Hisanari Otsu

Killian Herveau

Johannes Hanika

Carsten Dachsbacher

The design of the proposal distributions, and most notably the kernel parameters, are crucial for the performance of Markov chain Monte Carl… (see more)o (MCMC) rendering. A poor selection of parameters can increase the correlation of the Markov chain and result in bad rendering performance. We approach this problem by a novel path perturbation strategy for online-learning of state-dependent kernel parameters. We base our approach on the theoretical framework of regional adaptive MCMC which enables the adaptation of parameters depending on the region of the state space which contains the current sample, and on information collected from previous samples. For this, we define a partitioning of the path space on a low-dimensional canonical space to capture the characteristics of paths, with a focus on path segments closer to the sensor. Fast convergence is achieved by adaptive refinement of the partitions. Exemplarily, we present two novel regional adaptive path perturbation techniques akin to lens and multi-chain perturbations. Our approach can easily be used on top of existing path space MLT methods to improve rendering efficiency, while being agnostic to the initial choice of kernel parameters.

2024-02-13

ArXiv (preprint)

Minimax Exploiter: A Data Efficient Approach for Competitive Self-Play

Daniel Bairamian

Philippe Marcotte

Joshua Romoff

Gabriel Robert

2024-01-01

AAMAS (published)

A Model-Based Solution to the Offline Multi-Agent Reinforcement Learning Coordination Problem

Paul Barde

Jakob Nicolaus Foerster

Amy Zhang

2024-01-01

AAMAS (published)

Cone-Traced Supersampling with Subpixel Edge Reconstruction.

Andrei Chubarau

Yangyang Zhao

Ruby Rao

Paul Kry

While signed distance fields (SDFs) in theory offer infinite level of detail, they are typically rendered using the sphere tracing algorithm… (see more) at finite resolutions, which causes the common rasterized image synthesis problem of aliasing. Most existing optimized antialiasing solutions rely on polygon mesh representations; SDF-based geometry can only be directly antialiased with the computationally expensive supersampling or with post-processing filters that may produce undesirable blurriness and ghosting. In this work, we present cone-traced supersampling (CTSS), an efficient and robust spatial antialiasing solution that naturally complements the sphere tracing algorithm, does not require casting additional rays per pixel or offline prefiltering, and can be easily implemented in existing real-time SDF renderers. CTSS performs supersampling along the traced ray near surfaces with partial visibility – object contours – identified by evaluating cone intersections within a pixel's view frustum. We further introduce subpixel edge reconstruction (SER), a technique that extends CTSS to locate and resolve complex pixels with geometric edges in relatively flat regions, which are otherwise undetected by cone intersections. Our combined solution relies on a specialized sampling strategy to minimize the number of shading computations and correlates sample visibility to aggregate the samples. With comparable antialiasing quality at significantly lower computational cost, CTSS is a reliable practical alternative to conventional supersampling.

2023-12-14

IEEE Transactions on Visualization and Computer Graphics (published)

Efficient Graphics Representation with Differentiable Indirection

Sayantan Datta

Carl Marshall

Zhao Dong

Zhengqin Li

We introduce differentiable indirection – a novel learned primitive that employs differentiable multi-scale lookup tables as an effective … (see more)substitute for traditional compute and data operations across the graphics pipeline. We demonstrate its flexibility on a number of graphics tasks, i.e., geometric and image representation, texture mapping, shading, and radiance field representation. In all cases, differentiable indirection seamlessly integrates into existing architectures, trains rapidly, and yields both versatile and efficient results.

2023-12-11

SIGGRAPH Asia 2023 Conference Papers (published)

Differentiable visual computing for inverse problems and machine learning

Andrew Spielberg

Fangcheng Zhong

Konstantinos Rematas

Krishna Murthy

Cengiz Oztireli

Tzu-Mao Li

2023-11-17

Nature Machine Intelligence (published)

Learning Neural Implicit Representations with Surface Signal Parameterizations

Yanran Guan

Andrei Chubarau

Ruby Rao

2023-08-01

Computers & Graphics (published)

Parameter-space ReSTIR for Differentiable and Inverse Rendering

Wesley Chang

Venkataram Sivaram

Toshiya Hachisuka

Ravi Ramamoorthi

Tzu-Mao Li

Differentiable rendering is frequently used in gradient descent-based inverse rendering pipelines to solve for scene parameters – such as … (see more)reflectance or lighting properties – from target image inputs. Efficient computation of accurate, low variance gradients is critical for rapid convergence. While many methods employ variance reduction strategies, they operate independently on each gradient descent iteration, requiring large sample counts and computation. Gradients may however vary slowly between iterations, leading to unexplored potential benefits when reusing sample information to exploit this coherence. We develop an algorithm to reuse Monte Carlo gradient samples between gradient iterations, motivated by reservoir-based temporal importance resampling in forward rendering. Direct application of this method is not feasible, as we are computing many derivative estimates (i.e., one per optimization parameter) instead of a single pixel intensity estimate; moreover, each of these gradient estimates can affect multiple pixels, and gradients can take on negative values. We address these challenges by reformulating differential rendering integrals in parameter space, developing a new resampling estimator that treats negative functions, and combining these ideas into a reuse algorithm for inverse texture optimization. We significantly reduce gradient error compared to baselines, and demonstrate faster inverse rendering convergence in settings involving complex direct lighting and material textures.

2023-07-23

Special Interest Group on Computer Graphics and Interactive Techniques Conference Conference Proceedings (published)