Siamak Ravanbakhsh

Laurence Perreault-Levasseur

Biography

Siamak Ravanbakhsh is an assistant professor at McGill University’s School of Computer Science and a core academic member of Mila – Quebec Artificial Intelligence Institute.

Before joining McGill and Mila, he held a similar position at the University of British Columbia. Prior to that, he was a postdoctoral fellow at the Machine Learning Department and Robotics Institute of Carnegie Mellon University. He completed his PhD at the University of Alberta.

Ravanbakhsh’s research is centred around problems of representation learning, in particular the principled use of geometry, probabilistic inference and symmetry.

Current Students

Tara Akhound-Sadegh

PhD - McGill University

Co-supervisor :

Noah El Rimawi-Fine

Master's Research - McGill University

Principal supervisor :

Mathieu Blanchette

Qianyi Gao

Master's Research - McGill University

Daria Goptsii Goptsii

Professional Master's - McGill University

Vineet Jain

PhD - McGill University

Oumar Kaba

PhD - McGill University

Website

Daniel Levy

PhD - McGill University

Xiusi Li

PhD - McGill University

Master's Research - McGill University

Khang Ngo

PhD - McGill University

David Nitchi

Collaborating researcher - McGill University

Mohammad Pedramfar

Postdoctorate - McGill University

Kusha Sareen

Master's Research - McGill University

Website

Laurence Perreault-Levasseur

PhD - McGill University

Collaborating Alumni - McGill University

Publications

Lie Point Symmetry and Physics Informed Networks

Tara Akhound-Sadegh

Johannes Brandstetter

MAX WELLING

Symmetries have been leveraged to improve the generalization of neural networks through different mechanisms from data augmentation to equiv… (see more)ariant architectures. However, despite their potential, their integration into neural solvers for partial differential equations (PDEs) remains largely unexplored. We explore the integration of PDE symmetries, known as Lie point symmetries, in a major family of neural solvers known as physics-informed neural networks (PINNs). We propose a loss function that informs the network about Lie point symmetries in the same way that PINN models try to enforce the underlying PDE through a loss function. Intuitively, our symmetry loss ensures that the infinitesimal generators of the Lie group conserve the PDE solutions. Effectively, this means that once the network learns a solution, it also learns the neighbouring solutions generated by Lie point symmetries. Empirical evaluations indicate that the inductive bias introduced by the Lie point symmetries of the PDEs greatly boosts the sample efficiency of PINNs.

2023-09-20

NeurIPS.cc/2023/Conference (poster)

openreview.net

Using Multiple Vector Channels Improves E(n)-Equivariant Graph Neural Networks

Daniel Levy

Sékou-Oumar Kaba

Carmelo Gonzales

Santiago Miret

2023-09-05

ArXiv (preprint)

Equivariance with Learned Canonicalization Functions

Sékou-Oumar Kaba

Arnab Kumar Mondal

Yan Zhang

Yoshua Bengio

Symmetry-based neural networks often constrain the architecture in order to achieve invariance or equivariance to a group of transformations… (see more). In this paper, we propose an alternative that avoids this architectural constraint by learning to produce canonical representations of the data. These canonicalization functions can readily be plugged into non-equivariant backbone architectures. We offer explicit ways to implement them for some groups of interest. We show that this approach enjoys universality while providing interpretable insights. Our main hypothesis, supported by our empirical results, is that learning a small neural network to perform canonicalization is better than using predefined heuristics. Our experiments show that learning the canonicalization function is competitive with existing techniques for learning equivariant functions across many tasks, including image classification,

2023-07-02

Proceedings of the 40th International Conference on Machine Learning (published)

Characterization of inpaint residuals in interferometric measurements of the epoch of reionization

Michael Pagano

Jing Liu

Adrian Liu

Nicholas S. Kern

Aaron Ewall-Wice

Philip Bull

Robert Pascua

Zara Abdurashidova

Tyrone Adams

James E. Aguirre

Paul Alexander

Zaki S. Ali

Rushelle Baartman

Yanga Balfour

Adam P. Beardsley

Gianni Bernardi

Tashalee S. Billings

Judd D. Bowman

Richard F. Bradley … (see 58 more)

Jacob Burba

Steven Carey

Chris L. Carilli

Carina Cheng

David R. DeBoer

Eloy de Lera Acedo

Matt Dexter

Joshua S. Dillon

Nico Eksteen

John Ely

Nicolas Fagnoni

Randall Fritz

Steven R. Furlanetto

Kingsley Gale-Sides

Brian Glendenning

Deepthi Gorthi

Bradley Greig

Jasper Grobbelaar

Ziyaad Halday

Bryna J. Hazelton

Jacqueline N. Hewitt

Jack Hickish

Daniel C. Jacobs

Austin Julius

MacCalvin Kariseb

Joshua Kerrigan

Piyanat Kittiwisit

Saul A. Kohn

Matthew Kolopanis

Adam Lanman

Paul La Plante

Anita Loots

David Harold Edward MacMahon

Lourence Malan

Cresshim Malgas

Keith Malgas

Bradley Marero

Zachary E. Martinot

Andrei Mesinger

Mathakane Molewa

Miguel F. Morales

Tshegofalang Mosiane

Abraham R. Neben

Bojan Nikolic

Hans Nuwegeld

Aaron R. Parsons

Nipanjana Patra

Samantha Pieterse

Nima Razavi-Ghods

James Robnett

Kathryn Rosie

Peter Sims

Craig Smith

Hilton Swarts

Nithyanandan Thyagarajan

Pieter van Wyngaarden

Peter K. G. Williams

Haoxuan Zheng

Radio Frequency Interference (RFI) is one of the systematic challenges preventing 21cm interferometric instruments from detecting the Epoch … (see more)of Reionization. To mitigate the effects of RFI on data analysis pipelines, numerous inpaint techniques have been developed to restore RFI corrupted data. We examine the qualitative and quantitative errors introduced into the visibilities and power spectrum due to inpainting. We perform our analysis on simulated data as well as real data from the Hydrogen Epoch of Reionization Array (HERA) Phase 1 upper limits. We also introduce a convolutional neural network that capable of inpainting RFI corrupted data in interferometric instruments. We train our network on simulated data and show that our network is capable at inpainting real data without requiring to be retrained. We find that techniques that incorporate high wavenumbers in delay space in their modeling are best suited for inpainting over narrowband RFI. We also show that with our fiducial parameters Discrete Prolate Spheroidal Sequences (DPSS) and CLEAN provide the best performance for intermittent ``narrowband'' RFI while Gaussian Progress Regression (GPR) and Least Squares Spectral Analysis (LSSA) provide the best performance for larger RFI gaps. However we caution that these qualitative conclusions are sensitive to the chosen hyperparameters of each inpainting technique. We find these results to be consistent in both simulated and real visibilities. We show that all inpainting techniques reliably reproduce foreground dominated modes in the power spectrum. Since the inpainting techniques should not be capable of reproducing noise realizations, we find that the largest errors occur in the noise dominated delay modes. We show that in the future, as the noise level of the data comes down, CLEAN and DPSS are most capable of reproducing the fine frequency structure in the visibilities of HERA data.

2023-02-09

Monthly Notices of the Royal Astronomical Society (published)

Galaxies on graph neural networks: towards robust synthetic galaxy catalogs with deep generative models

Yesukhei Jagvaral

François Lanusse

Sukhdeep Singh

Rachel Mandelbaum

Duncan Campbell

The future astronomical imaging surveys are set to provide precise constraints on cosmological parameters, such as dark energy. However, pro… (see more)-duction of synthetic data for these surveys, to test and validate analysis methods, suﬀers from a very high computational cost. In particular, generating mock galaxy catalogs at suﬃciently large volume and high resolution will soon become computa-tionally unreachable. In this paper, we address this problem with a Deep Generative Model to create robust mock galaxy catalogs that may be used to test and develop the analysis pipelines of future weak lensing surveys. We build our model on a custom built Graph Convolutional Networks, by placing each galaxy on a graph node and then connecting the graphs within each gravitationally bound system. We train our model on a cosmological simulation with realistic galaxy populations to capture the 2D and 3D orientations of galaxies. The samples from the model exhibit comparable statistical properties to those in the simulations. To the best of our knowledge, this is the ﬁrst instance of a generative model on graphs in an astrophysical/cosmological context.

2022-12-10

ArXiv (preprint)

Galaxies and Halos on Graph Neural Networks: Deep Generative Modeling Scalar and Vector Quantities for Intrinsic Alignment

Yesukhei Jagvaral

François Lanusse

Sukhdeep Singh

Rachel Mandelbaum

Duncan Campbell

In order to prepare for the upcoming wide-field cosmological surveys, large simulations of the Universe with realistic galaxy populations ar… (see more)e required. In particular, the tendency of galaxies to naturally align towards overdensities, an effect called intrinsic alignments (IA), can be a major source of systematics in the weak lensing analysis. As the details of galaxy formation and evolution relevant to IA cannot be simulated in practice on such volumes, we propose as an alternative a Deep Generative Model. This model is trained on the IllustrisTNG-100 simulation and is capable of sampling the orientations of a population of galaxies so as to recover the correct alignments. In our approach, we model the cosmic web as a set of graphs, where the graphs are constructed for each halo, and galaxy orientations as a signal on those graphs. The generative model is implemented on a Generative Adversarial Network architecture and uses specifically designed Graph-Convolutional Networks sensitive to the relative 3D positions of the vertices. Given (sub)halo masses and tidal fields, the model is able to learn and predict scalar features such as galaxy and dark matter subhalo shapes; and more importantly, vector features such as the 3D orientation of the major axis of the ellipsoid and the complex 2D ellipticities. For correlations of 3D orientations the model is in good quantitative agreement with the measured values from the simulation, except for at very small and transition scales. For correlations of 2D ellipticities, the model is in good quantitative agreement with the measured values from the simulation on all scales. Additionally, the model is able to capture the dependence of IA on mass, morphological type and central/satellite type.

2022-08-01

Monthly Notices of the Royal Astronomical Society (unknown)

EqR: Equivariant Representations for Data-Efficient Reinforcement Learning

Arnab Kumar Mondal

Vineet Jain

Kaleem Siddiqi

2022-06-27

Proceedings of the 39th International Conference on Machine Learning (published)

Utility Theory for Sequential Decision Making

The von Neumann-Morgenstern (VNM) utility theorem shows that under certain axioms of rationality, decision-making is reduced to maximizing t… (see more)he expectation of some utility function. We extend these axioms to increasingly structured sequential decision making settings and identify the structure of the corresponding utility functions. In particular, we show that memoryless preferences lead to a utility in the form of a per transition reward and multiplicative factor on the future return. This result motivates a generalization of Markov Decision Processes (MDPs) with this structure on the agent's returns, which we call Affine-Reward MDPs. A stronger constraint on preferences is needed to recover the commonly used cumulative sum of scalar rewards in MDPs. A yet stronger constraint simplifies the utility function for goal-seeking agents in the form of a difference in some function of states that we call potential functions. Our necessary and sufficient conditions demystify the reward hypothesis that underlies the design of rational agents in reinforcement learning by adding an axiom to the VNM rationality axioms and motivates new directions for AI research involving sequential decision making.

2022-06-27

Proceedings of the 39th International Conference on Machine Learning (published)

Transformation Coding: Simple Objectives for Equivariant Representations

Arnab Kumar Mondal

2022-02-18

ArXiv (preprint)

Equivariant Networks for Crystal Structures

Sékou-Oumar Kaba

Supervised learning with deep models has tremendous potential for applications in materials science. Recently, graph neural networks have be… (see more)en used in this context, drawing direct inspiration from models for molecules. However, materials are typically much more structured than molecules, which is a feature that these models do not leverage. In this work, we introduce a class of models that are equivariant with respect to crystalline symmetry groups. We do this by defining a generalization of the message passing operations that can be used with more general permutation groups, or that can alternatively be seen as defining an expressive convolution operation on the crystal graph. Empirically, these models achieve competitive results with state-of-the-art on property prediction tasks.

2021-12-31

Advances in Neural Information Processing Systems 35 (NeurIPS 2022) (published)

openreview.net

Structuring Representations Using Group Invariants

Arnab Kumar Mondal

2021-12-31

Advances in Neural Information Processing Systems 35 (NeurIPS 2022) (published)

openreview.net

Equivariant Networks for Pixelized Spheres

Pixelizations of Platonic solids such as the cube and icosahedron have been widely used to represent spherical data, from climate records to… (see more) Cosmic Microwave Background maps. Platonic solids have well-known global symmetries. Once we pixelize each face of the solid, each face also possesses its own local symmetries in the form of Euclidean isometries. One way to combine these symmetries is through a hierarchy. However, this approach does not adequately model the interplay between the two levels of symmetry transformations. We show how to model this interplay using ideas from group theory, identify the equivariant linear maps, and introduce equivariant padding that respects these symmetries. Deep networks that use these maps as their building blocks generalize gauge equivariant CNNs on pixelized spheres. These deep networks achieve state-of-the-art results on semantic segmentation for climate data and omnidirectional image processing. Code is available at https://git.io/JGiZA.

2021-06-30

Proceedings of the 38th International Conference on Machine Learning (published)