Derek Nowrouzezahrai

Membre académique principal

derek@mila.quebec

Chaire en IA Canada-CIFAR

Professeur titulaire, McGill University, Département de génie électrique et informatique

Sujets de recherche

Apprentissage de représentations

Apprentissage par renforcement

Apprentissage profond

Modèles génératifs

Photographie computationnelle

Systèmes dynamiques

Vision par ordinateur

Google Scholar

Biographie

Derek Nowrouzezahrai est professeur titulaire à l'Université McGill, directeur du Centre sur les machines intelligentes et codirecteur du Laboratoire de graphisme et d’imagerie de McGill (MGIL), ainsi que titulaire d’une chaire en IA Canada-CIFAR et de la chaire Ubisoft-Mila de mise à l'échelle des univers de jeux grâce à une IA responsable.

Ses recherches portent sur la simulation de divers phénomènes physiques - tels que la dynamique des objets en mouvement et l'éclairage pour la synthèse d'images réalistes - avec des applications dans les domaines de la réalité virtuelle, des jeux vidéo, de la simulation fluide et contrôlée, de la fabrication numérique, de l'optique augmentée par le calcul et du traitement de la géométrie. En outre, Derek s'intéresse au développement de simulateurs dérivables de ces systèmes dynamiques et à leurs applications aux problèmes inverses en robotique et dans le domaine de la vision.

Son travail repose sur le développement de méthodes Monte Carlo à haute performance et efficaces en matière d'échantillonnage (chaîne de Markov), de statistiques d'ordre élevé et de méthodes de calcul pour les problèmes d'intégration multidimensionnelle complexes, de simulateurs dérivables basés sur la physique et de méthodes numériques pour les systèmes dynamiques, ainsi que sur l'application de l'apprentissage automatique aux médias 3D, visuels et interactifs.

Étudiants actuels

Daniel Bairamian

Doctorat - McGill

daniel.bairamian@mila.quebec

Paul Barde

Collaborateur·rice de recherche - McGill

Co-superviseur⋅e :

Chris Pal

paul.b.barde@gmail.com

Doctorat - McGill

Co-superviseur⋅e :

Kaleem Siddiqi

shubham.chopra@mail.mcgill.ca

Github

Chris Emezue

Doctorat - UdeM

Superviseur⋅e principal⋅e :

Chris Pal

chris.emezue@mila.quebec

Doctorat - McGill

chenbo.jiang@mail.mcgill.ca

Xiangyu (Daniel) Kong

Doctorat - McGill

xiangyu.kong@mila.quebec

Github

Joey Marten

Maîtrise recherche - McGill

Co-superviseur⋅e :

Noam Aigerman

joey.marten@mila.quebec

Github

Hossam Mohamed S M Ahmed

Doctorat - McGill

hossam.mohamed@mila.quebec

Github

Somjit Nath

Doctorat - McGill

Superviseur⋅e principal⋅e :

Samira Ebrahimi Kahou

somjit.nath@mila.quebec

Site web

Google Scholar

Yasaman Sabbagh Ziarani

Doctorat - McGill

yasaman.sabbagh@mila.quebec

Yuanyuan Tao

Doctorat - McGill

yuanyuan.tao@mila.quebec

Mattie Tesfaldet

Collaborateur·rice de recherche - McGill

Co-superviseur⋅e :

Chris Pal

mattie.tesfaldet@mila.quebec

Site web

Github

Google Scholar

Junha Yoo

Maîtrise recherche - McGill

Co-superviseur⋅e :

James Richard Forbes

junha.yoo@mila.quebec

Billets de blogue

An illustration of the forward and backward processes of a differentiable renderer.

29 janvier 2025

Informatique visuelle différentiable : passer de la deuxième à la troisième dimension dans les applications d’apprentissage automatique

par

Jordan J. Bannister

Derek Nowrouzezahrai

Lire l'article

Publications

Overcoming Challenges in Leveraging GANs for Few-Shot Data Augmentation

Pau Rodríguez

Christopher Pal

In this paper, we explore the use of GAN-based few-shot data augmentation as a method to improve few-shot classification performance. We per… (voir plus)form an exploration into how a GAN can be fine-tuned for such a task (one of which is in a class-incremental manner), as well as a rigorous empirical investigation into how well these models can perform to improve few-shot classification. We identify issues related to the difficulty of training such generative models under a purely supervised regime with very few examples, as well as issues regarding the evaluation protocols of existing works. We also find that in this regime, classification accuracy is highly sensitive to how the classes of the dataset are randomly split. Therefore, we propose a semi-supervised fine-tuning approach as a more pragmatic way forward to address these problems.

2022-11-27

Proceedings of The 1st Conference on Lifelong Learning Agents (publié)

doi.org

proceedings.mlr.press

Learning Latent Structural Causal Models

Jithendaraa Subramanian

Yashas Annadani

Ivaxi Sheth

Nan Rosemary Ke

Tristan Deleu

Stefan Bauer

Derek Nowrouzezahrai

Samira Ebrahimi Kahou

Causal learning has long concerned itself with the accurate recovery of underlying causal mechanisms. Such causal modelling enables better e… (voir plus)xplanations of out-of-distribution data. Prior works on causal learning assume that the high-level causal variables are given. However, in machine learning tasks, one often operates on low-level data like image pixels or high-dimensional vectors. In such settings, the entire Structural Causal Model (SCM) -- structure, parameters, \textit{and} high-level causal variables -- is unobserved and needs to be learnt from low-level data. We treat this problem as Bayesian inference of the latent SCM, given low-level data. For linear Gaussian additive noise SCMs, we present a tractable approximate inference method which performs joint inference over the causal variables, structure and parameters of the latent SCM from random, known interventions. Experiments are performed on synthetic datasets and a causally generated image dataset to demonstrate the efficacy of our approach. We also perform image generation from unseen interventions, thereby verifying out of distribution generalization for the proposed causal model.

2022-10-23

arXiv (prépublication)

doi.org

openreview.net

Learning to Guide and to Be Guided in the Architect-Builder Problem

Paul Barde

Tristan Karch

Derek Nowrouzezahrai

Clément Moulin-Frier

Christopher Pal

Pierre-Yves Oudeyer

We are interested in interactive agents that learn to coordinate, namely, a …

2022-04-24

International Conference on Learning Representations (Accept (Poster))

doi.org

openreview.net

Adversarial Soft Advantage Fitting: Imitation Learning without Policy Optimization

Christopher Pal

Adversarial Imitation Learning alternates between learning a discriminator -- which tells apart expert's demonstrations from generated ones … (voir plus)-- and a generator's policy to produce trajectories that can fool this discriminator. This alternated optimization is known to be delicate in practice since it compounds unstable adversarial training with brittle and sample-inefficient reinforcement learning. We propose to remove the burden of the policy optimization steps by leveraging a novel discriminator formulation. Specifically, our discriminator is explicitly conditioned on two policies: the one from the previous generator's iteration and a learnable policy. When optimized, this discriminator directly learns the optimal generator's policy. Consequently, our discriminator's update solves the generator's optimization problem for free: learning a policy that imitates the expert does not require an additional optimization loop. This formulation effectively cuts by half the implementation and computational burden of Adversarial Imitation Learning algorithms by removing the Reinforcement Learning phase altogether. We show on a variety of tasks that our simpler approach is competitive to prevalent Imitation Learning methods.

2020-12-06

Conference on Neural Information Processing Systems (Accept (poster))

doi.org

arxiv.org

Navigation Agents for the Visually Impaired: A Sidewalk Simulator and Experiments

Martin Weiss

Simon Chamorro

Roger Girgis

Margaux Luck

Samira E. Kahou

Joseph P. Cohen

Millions of blind and visually-impaired (BVI) people navigate urban environments every day, using smartphones for high-level path-planning a… (voir plus)nd white canes or guide dogs for local information. However, many BVI people still struggle to travel to new places. In our endeavor to create a navigation assistant for the BVI, we found that existing Reinforcement Learning (RL) environments were unsuitable for the task. This work introduces SEVN, a sidewalk simulation environment and a neural network-based approach to creating a navigation agent. SEVN contains panoramic images with labels for house numbers, doors, and street name signs, and formulations for several navigation tasks. We study the performance of an RL algorithm (PPO) in this setting. Our policy model fuses multi-modal observations in the form of variable resolution images, visible text, and simulated GPS data to navigate to a goal door. We hope that this dataset, simulator, and experimental results will provide a foundation for further research into the creation of agents that can assist members of the BVI community with outdoor navigation.

2020-05-11

Proceedings of the Conference on Robot Learning (publié)

doi.org

proceedings.mlr.press

Pix2Shape – Towards Unsupervised Learning of 3D Scenes from Images using a View-based Representation

Sai Rajeswar

Fahim Mannan

Florian Golemo

Jérôme Parent-lévesque

David Vázquez

Derek Nowrouzezahrai

Aaron Courville

We infer and generate three-dimensional (3D) scene information from a single input image and without supervision. This problem is under-expl… (voir plus)ored, with most prior work relying on supervision from, e.g., 3D ground-truth, multiple images of a scene, image silhouettes or key-points. We propose Pix2Shape, an approach to solve this problem with four components: (i) an encoder that infers the latent 3D representation from an image, (ii) a decoder that generates an explicit 2.5D surfel-based reconstruction of a scene from the latent code (iii) a differentiable renderer that synthesizes a 2D image from the surfel representation, and (iv) a critic network trained to discriminate between images generated by the decoder-renderer and those from a training distribution. Pix2Shape can generate complex 3D scenes that scale with the view-dependent on-screen resolution, unlike representations that capture world-space resolution, i.e., voxels or meshes. We show that Pix2Shape learns a consistent scene representation in its encoded latent space and that the decoder can then be applied to this latent representation in order to synthesize the scene from a novel viewpoint. We evaluate Pix2Shape with experiments on the ShapeNet dataset as well as on a novel benchmark we developed, called 3D-IQTT, to evaluate models based on their ability to enable 3d spatial reasoning. Qualitative and quantitative evaluation demonstrate Pix2Shape's ability to solve scene reconstruction, generation, and understanding tasks.

2020-03-19

International Journal of Computer Vision (inconnu)

doi.org

arxiv.org

Sidewalk Environment for Visual Navigation

Martin Weiß

Simon Chamorro

Roger Girgis

Margaux Luck

Samira Ebrahimi Kahou

Joseph Cohen

This dataset contains low and high resolution panoramic images, coordinates, labels and a connectivity graph. In order to run this simulated… (voir plus) environment, you will need at least one copy of the panoramic images, which are available in low (84x224 pixels) or high resolution (1280x3840 pixels). For more information, visit https://mweiss17.github.io/SEVN/.

2019-11-06

Polytechnique Montréal (accepté)

doi.org

Pix2Scene: Learning Implicit 3D Representations from Images

Sai Rajeswar

Fahim Mannan

2018-09-26

(publié)