Portrait of Derek Nowrouzezahrai

Derek Nowrouzezahrai

Core Academic Member
derek@mila.quebec
Canada CIFAR AI Chair
Associate Professor, McGill University, Department of Electrical and Computer Engineering
Research Topics
Computational Photography
Computer Vision
Deep Learning
Dynamical Systems
Generative Models
Reinforcement Learning
Representation Learning
Google Scholar

Biography

Derek Nowrouzezahrai is a full professor at McGill University, where he directs the Centre for Intelligent Machines and co-directs the Graphics Lab.

He is also a Canada CIFAR AI Chair and holds the Ubisoft–Mila research Chair, Scaling Game Worlds with Responsible AI.

Nowrouzezahrai’s research tackles the simulation of various physical phenomena, such as the dynamics of moving objects and the simulation of lighting for realistic image synthesis, which have applications in virtual reality, video games, fluid simulation and control, digital manufacturing, computationally augmented optics and geometry processing. He is also interested in the development of differentiable simulators of these dynamical systems and their applications to inverse problems in robotics and vision.

This work relies fundamentally on developing high performance and sample efficient (Markov chain) Monte Carlo-based methods, high-order statistics and computational methods for complex multi-dimensional integration problems, differentiable physics-based simulators and numerical methods for dynamical systems, and on applying machine learning to 3D, visual and interactive media.

Current Students

Daniel Bairamian
PhD - McGill University
Paul Barde
PhD - McGill University
Co-supervisor :
Chris Pal
paul.b.barde@gmail.com
Website
Github
Google Scholar
Chris Emezue
Master's Research - Université de Montréal
Principal supervisor :
Chris Pal
Website
Github
Google Scholar
Florian Golemo
Postdoctorate - McGill University
Co-supervisor :
Gauthier Gidel
Website
Github
Google Scholar
Daniel Kong
PhD - McGill University
Github
Ken Ming Lee
Master's Research - McGill University
Website
Github
Joey Litalien
PhD - McGill University
Website
Github
Google Scholar
Hossam Mohamed S M Ahmed
PhD - McGill University
Github
Somjit Nath
PhD - McGill University
Principal supervisor :
Samira Ebrahimi Kahou
Website
Google Scholar
Tooba Rahimnia
Master's Research - McGill University
Yasaman Sabbagh Ziarani
PhD - McGill University
Jithendaraa Subramanian
Master's Research - McGill University
Co-supervisor :
Samira Ebrahimi Kahou
Website
Github
Google Scholar
Yuanyuan Tao
PhD - McGill University
Mattie Tesfaldet
PhD - McGill University
Co-supervisor :
Chris Pal
Website
Github
Google Scholar

Publications

MeshDiffusion: Score-based Generative 3D Mesh Modeling
Zhen Liu
Yao Feng
Michael J. Black
Derek Nowrouzezahrai
Liam Paull
Weiyang Liu
We consider the task of generating realistic 3D shapes, which is useful for a variety of applications such as automatic scene generation and… (see more) physical simulation. Compared to other 3D representations like voxels and point clouds, meshes are more desirable in practice, because (1) they enable easy and arbitrary manipulation of shapes for relighting and simulation, and (2) they can fully leverage the power of modern graphics pipelines which are mostly optimized for meshes. Previous scalable methods for generating meshes typically rely on sub-optimal post-processing, and they tend to produce overly-smooth or noisy surfaces without fine-grained geometric details. To overcome these shortcomings, we take advantage of the graph structure of meshes and use a simple yet very effective generative modeling method to generate 3D meshes. Specifically, we represent meshes with deformable tetrahedral grids, and then train a diffusion model on this direct parameterization. We demonstrate the effectiveness of our model on multiple generative tasks.
2023-02-01
ICLR.cc/2023/Conference (notable)
doi.org
openreview.net
From Words to Blocks: Building Objects by Grounding Language Models with Reinforcement Learning
Michael Ahn
Anthony Brohan
Noah Brown
liang Dai
Dan Su
Holy Lovenia Ziwei Bryan Wilie
Tiezheng Yu
Willy Chung
Quyet V. Do
Paul Barde
Tristan Karch
Derek Nowrouzezahrai
C. Bonial
Mitchell Abrams
David R. Traum
Hyung Won
Le Hou
Shayne Longpre
Yi Zoph
William Tay … (see 32 more)
Eric Fedus
Xuezhi Li
Lasse Espeholt
Hubert Soyer
Remi Munos
Karen Si-801
Vlad Mnih
Tom Ward
Yotam Doron
Wenlong Huang
Pieter Abbeel
Deepak Pathak
Julia Kiseleva
Ziming Li
Mohammad Aliannejadi
Shrestha Mohanty
Maartje Ter Hoeve
Mikhail Burtsev
Alexey Skrynnik
Artem Zholus
A. Panov
Kavya Srinet
A. Szlam
Yuxuan Sun
Katja Hofmann
Marc-Alexandre Côté
Ahmed Hamid Awadallah
Linar Abdrazakov
Igor Churin
Putra Manggala
Kata Naszádi
Michiel Van Der Meer
Leveraging pre-trained language models to gen-001 erate action plans for embodied agents is an 002 emerging research direction. However, exe… (see more)-003 cuting instructions in real or simulated envi-004 ronments necessitates verifying the feasibility 005 of actions and their relevance in achieving a 006 goal. We introduce a novel method that in-007 tegrates a language model and reinforcement 008 learning for constructing objects in a Minecraft-009 like environment, based on natural language 010 instructions. Our method generates a set of 011 consistently achievable sub-goals derived from 012 the instructions and subsequently completes the 013 associated sub-tasks using a pre-trained RL pol-014 icy. We employ the IGLU competition, which 015 is based on the Minecraft-like simulator, as our 016 test environment, and compare our approach 017 to the competition’s top-performing solutions. 018 Our approach outperforms existing solutions in 019 terms of both the quality of the language model 020 and the quality of the structures built within the 021 IGLU environment. 022
2023-01-01
(published)
www.semanticscholar.org
Learning Latent Structural Causal Models
Jithendaraa Subramanian
Yashas Annadani
Ivaxi Sheth
Nan Rosemary Ke
Tristan Deleu
Stefan Bauer
Derek Nowrouzezahrai
Samira Ebrahimi Kahou
Causal learning has long concerned itself with the accurate recovery of underlying causal mechanisms. Such causal modelling enables better e… (see more)xplanations of out-of-distribution data. Prior works on causal learning assume that the high-level causal variables are given. However, in machine learning tasks, one often operates on low-level data like image pixels or high-dimensional vectors. In such settings, the entire Structural Causal Model (SCM) -- structure, parameters, \textit{and} high-level causal variables -- is unobserved and needs to be learnt from low-level data. We treat this problem as Bayesian inference of the latent SCM, given low-level data. For linear Gaussian additive noise SCMs, we present a tractable approximate inference method which performs joint inference over the causal variables, structure and parameters of the latent SCM from random, known interventions. Experiments are performed on synthetic datasets and a causally generated image dataset to demonstrate the efficacy of our approach. We also perform image generation from unseen interventions, thereby verifying out of distribution generalization for the proposed causal model.
2022-10-24
ArXiv (preprint)
doi.org
openreview.net
Single‐pass stratified importance resampling
Ege Ciklabakkal
Adrien Gruson
Iliyan Georgiev
Derek Nowrouzezahrai
Toshiya Hachisuka
Resampling is the process of selecting from a set of candidate samples to achieve a distribution (approximately) proportional to a desired t… (see more)arget. Recent work has revisited its application to Monte Carlo integration, yielding powerful and practical importance sampling methods. One drawback of existing resampling methods is that they cannot generate stratified samples. We propose two complementary techniques to achieve efficient stratified resampling. We first introduce bidirectional CDF sampling which yields the same result as conventional inverse CDF sampling but in a single pass over the candidates, without needing to store them, similarly to reservoir sampling. We then order the candidates along a space‐filling curve to ensure that stratified CDF sampling of candidate indices yields stratified samples in the integration domain. We showcase our method on various resampling‐based rendering problems.
2022-07-30
Computer Graphics Forum (published)
doi.org
Kubric: A scalable dataset generator
Klaus Greff
Francois Belletti
Lucas Beyer
Carl Doersch
Yilun Du
Daniel Duckworth
David J Fleet
Dan Gnanapragasam
Florian Golemo
Charles Herrmann
Thomas Kipf
Abhijit Kundu
Dmitry Lagun
Issam Hadj Laradji
Hsueh-Ti Liu
Henning Meyer
Yishu Miao
Derek Nowrouzezahrai
Cengiz Oztireli
Etienne Pot … (see 14 more)
Noha Radwan
Daniel Rebain
Sara Sabour
Mehdi S. M. Sajjadi
Matan Sela
Vincent Sitzmann
Austin Stone
Deqing Sun
Suhani Vora
Ziyu Wang
Tianhao Wu
Kwang Moo Yi
Fangcheng Zhong
Andrea Tagliasacchi
Data is the driving force of machine learning, with the amount and quality of training data often being more important for the performance o… (see more)f a system than architecture and training details. But collecting, processing and annotating real data at scale is difficult, expensive, and frequently raises additional privacy, fairness and legal concerns. Synthetic data is a powerful tool with the potential to address these shortcomings: 1) it is cheap 2) supports rich ground-truth annotations 3) offers full control over data and 4) can circumvent or mitigate problems regarding bias, privacy and licensing. Unfortunately, software tools for effective data generation are less mature than those for architecture design and training, which leads to fragmented generation efforts. To address these problems we introduce Kubric, an open-source Python framework that interfaces with PyBullet and Blender to generate photo-realistic scenes, with rich annotations, and seamlessly scales to large jobs distributed over thousands of machines, and generating TBs of data. We demonstrate the effectiveness of Kubric by presenting a series of 13 different generated datasets for tasks ranging from studying 3D NeRF models to optical flow estimation. We release Kubric, the used assets, all of the generation code, as well as the rendered datasets for reuse and modification.
2022-06-18
2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (published)
doi.org
arxiv.org
Kubric: A scalable dataset generator
Klaus Greff
Francois Belletti
Lucas Beyer
Carl Doersch
Yilun Du
Daniel Duckworth
David J. Fleet
Dan Gnanapragasam
Florian Golemo
Charles Herrmann
Thomas N. Kipf
Abhijit Kundu
Dmitry Lagun
Issam Hadj Laradji
Hsueh-Ti Liu
H. Meyer
Yishu Miao
Derek Nowrouzezahrai
Cengiz Oztireli
Etienne Pot … (see 14 more)
Noha Radwan
Daniel Rebain
Sara Sabour
Mehdi S. M. Sajjadi
Matan Sela
Vincent Sitzmann
Austin Stone
Deqing Sun
Suhani Vora
Ziyu Wang
Tianhao Wu
Kwang Moo Yi
Fangcheng Zhong
Andrea Tagliasacchi
Data is the driving force of machine learning, with the amount and quality of training data often being more important for the performance o… (see more)f a system than architecture and training details. But collecting, processing and annotating real data at scale is difficult, expensive, and frequently raises additional privacy, fairness and legal concerns. Synthetic data is a powerful tool with the potential to address these shortcomings: 1) it is cheap 2) supports rich ground-truth annotations 3) offers full control over data and 4) can circumvent or mitigate problems regarding bias, privacy and licensing. Unfortunately, software tools for effective data generation are less mature than those for architecture design and training, which leads to fragmented generation efforts. To address these problems we introduce Kubric, an open-source Python framework that interfaces with PyBullet and Blender to generate photo-realistic scenes, with rich annotations, and seamlessly scales to large jobs distributed over thousands of machines, and generating TBs of data. We demonstrate the effectiveness of Kubric by presenting a series of 13 different generated datasets for tasks ranging from studying 3D NeRF models to optical flow estimation. We release Kubric, the used assets, all of the generation code, as well as the rendered datasets for reuse and modification.
2022-03-07
ArXiv (preprint)
doi.org
arxiv.org