David Rolnick

Biography

David Rolnick is an assistant professor at McGill University’s School of Computer Science, a core academic member of Mila – Quebec Artificial Intelligence Institute and holds a Canada CIFAR AI Chair. Rolnick’s work focuses on applications of machine learning to help address climate change. He is the co-founder and chair of Climate Change AI, and scientific co-director of Sustainability in the Digital Age. After completing his PhD in applied mathematics at the Massachusetts Institute of Technology (MIT), he was a NSF Mathematical Sciences Postdoctoral Research Fellow, an NSF Graduate Research Fellow and a Fulbright Scholar. He was named to MIT Technology Review’s “35 Innovators Under 35” in 2021.

Current Students

Aditya Aditya

Collaborating researcher

Collaborating Alumni - McGill University

Collaborating researcher - Cambridge University

Co-supervisor :

Postdoctorate - McGill University

Michael Bunsen

Collaborating researcher - McGill University

Shahana Chatterjee

Collaborating researcher - N/A

Co-supervisor :

Yoshua Bengio

Yuyan Chen

PhD - McGill University

Eya Cherif

Collaborating researcher - Leipzig University

Othmane Echchabi

Master's Research - McGill University

Collaborating researcher

Mohamed Elabbas

Collaborating researcher

Jannik Endres

Collaborating researcher

Jacopo Ghirri

Independent visiting researcher - Politecnico di Milano

Paula Harder

Independent visiting researcher

Collaborating researcher - Johannes Kepler University

Christina Isaicu Isaicu

Collaborating researcher - University of Amsterdam

Gaurav Iyer

Master's Research - McGill University

Julia Kaltenborn

PhD - McGill University

Devin Kwok

PhD - McGill University

Independent visiting researcher - Université de Montréal

Pierre-Louis Lemaire

Collaborating researcher - Polytechnique Montréal Montréal

Principal supervisor :

Alex Hernández-García

Joshi Manoj

Collaborating researcher - University of East Anglia

David Mickisch

Collaborating researcher

Juan Nathaniel Nathaniel

Collaborating researcher - Columbia university

Postdoctorate - McGill University

Co-supervisor :

Lena Podina

Collaborating researcher - University of Waterloo

Co-supervisor :

Yoshua Bengio

Venkatesh Ramesh

Collaborating Alumni - Université de Montréal

Marlena Reil

Master's Research - McGill University

Carla Roesch

Collaborating researcher - Columbia university

Rasha Saha

Master's Research - McGill University

Luca Marie Schmidt

Collaborating researcher - University of Tübingen

Independent visiting researcher - Karlsruhe Institute of Technology

Wietze Suijker

Independent visiting researcher

Ilija Trajković

Collaborating researcher - Karlsruhe Institute of Technology

Gabriel Tseng

PhD - McGill University

Donna Vakalis

Collaborating Alumni - Université de Montréal

Principal supervisor :

Collaborating researcher

Catherine Villeneuve

PhD - McGill University

Shan Zhao

Collaborating researcher - Technical University of Munich

Democratizing Access to Satellite Data with AI

Blog Posts

diagram illustrating how the AI foundation model for Earth observation, Galileo, works

October 21, 2025

Gabriel Tseng

David Rolnick

Read the article

Publications

Clustering units in neural networks: upstream vs downstream information

Richard D Lange

Konrad Paul Kording

It has been hypothesized that some form of"modular"structure in artificial neural networks should be useful for learning, compositionality, … (see more)and generalization. However, defining and quantifying modularity remains an open problem. We cast the problem of detecting functional modules into the problem of detecting clusters of similar-functioning units. This begs the question of what makes two units functionally similar. For this, we consider two broad families of methods: those that define similarity based on how units respond to structured variations in inputs ("upstream"), and those based on how variations in hidden unit activations affect outputs ("downstream"). We conduct an empirical study quantifying modularity of hidden layer representations of simple feedforward, fully connected networks, across a range of hyperparameters. For each model, we quantify pairwise associations between hidden units in each layer using a variety of both upstream and downstream measures, then cluster them by maximizing their"modularity score"using established tools from network science. We find two surprising results: first, dropout dramatically increased modularity, while other forms of weight regularization had more modest effects. Second, although we observe that there is usually good agreement about clusters within both upstream methods and downstream methods, there is little agreement about the cluster assignments across these two families of methods. This has important implications for representation-learning, as it suggests that finding modular representations that reflect structure in inputs (e.g. disentanglement) may be a distinct goal from learning modular representations that reflect structure in outputs (e.g. compositionality).

2022-06-12

TMLR (accepted)

On Neural Architecture Inductive Biases for Relational Tasks

Current deep learning approaches have shown good in-distribution generalization performance, but struggle with out-of-distribution generaliz… (see more)ation. This is especially true in the case of tasks involving abstract relations like recognizing rules in sequences, as we find in many intelligence tests. Recent work has explored how forcing relational representations to remain distinct from sensory representations, as it seems to be the case in the brain, can help artificial systems. Building on this work, we further explore and formalize the advantages afforded by 'partitioned' representations of relations and sensory details, and how this inductive bias can help recompose learned relational structure in newly encountered settings. We introduce a simple architecture based on similarity scores which we name Compositional Relational Network (CoRelNet). Using this model, we investigate a series of inductive biases that ensure abstract relations are learned and represented distinctly from sensory data, and explore their effects on out-of-distribution generalization for a series of relational psychophysics tasks. We find that simple architectural choices can outperform existing models in out-of-distribution generalization. Together, these results show that partitioning relational representations from other information streams may be a simple way to augment existing network architectures' robustness when performing out-of-distribution relational computations.

2022-06-08

ArXiv (preprint)

arxiv.org

Aligning artificial intelligence with climate change mitigation

Lynn H. Kaack

Priya L. Donti

Emma Strubell

George Kamiya

Felix Creutzig

2022-05-31

Nature Climate Change (unknown)

Inductive Biases for Relational Tasks

Blake Aaron Richards

Current deep learning approaches have shown good in-distribution performance but struggle in out-of-distribution settings. This is especiall… (see more)y true in the case of tasks involving abstract relations like recognizing rules in sequences, as required in many intelligence tests. In contrast, our brains are remarkably flexible at such tasks, an attribute that is likely linked to anatomical constraints on computations. Inspired by this, recent work has explored how enforcing that relational representations remain distinct from sensory representations can help artificial systems. Building on this work, we further explore and formalize the advantages afforded by ``partitioned'' representations of relations and sensory details. We investigate inductive biases that ensure abstract relations are learned and represented distinctly from sensory data across several neural network architectures and show that they outperform existing architectures on out-of-distribution generalization for various relational tasks. These results show that partitioning relational representations from other information streams may be a simple way to augment existing network architectures' robustness when performing relational computations.

2022-03-24

ICLR.cc/2022/Workshop/OSC (poster)

Tackling Climate Change with Machine Learning

Priya L. Donti

Lynn H. Kaack

Kelly Kochanski

Alexandre Lacoste

Kris Sankaran

Andrew Slavin Ross

Nikola Milojevic-Dupont

Natasha Jaques

Anna Waldman-Brown

Alexandra Luccioni

Tegan Maharaj

Evan D. Sherwin

S. Karthik Mukkavilli

Konrad P. Kording

Carla Gomes

Andrew Y. Ng

Demis Hassabis

John C. Platt

Felix Creutzig … (see 2 more)

Jennifer Chayes

Yoshua Bengio

Climate change is one of the greatest challenges facing humanity, and we, as machine learning experts, may wonder how we can help. Here we d… (see more)escribe how machine learning can be a powerful tool in reducing greenhouse gas emissions and helping society adapt to a changing climate. From smart grids to disaster management, we identify high impact problems where existing gaps can be filled by machine learning, in collaboration with other fields. Our recommendations encompass exciting research questions as well as promising business opportunities. We call on the machine learning community to join the global effort against climate change.

2022-02-06

ACM Computing Surveys (published)

arxiv.org

TIML: Task-Informed Meta-Learning for Agriculture

Gabriel Tseng

Hannah Kerner

Labeled datasets for agriculture are extremely spatially imbalanced. When developing algorithms for data-sparse regions, a natural approach … (see more)is to use transfer learning from data-rich regions. While standard transfer learning approaches typically leverage only direct inputs and outputs, geospatial imagery and agricultural data are rich in metadata that can inform transfer learning algorithms, such as the spatial coordinates of data-points or the class of task being learned. We build on previous work exploring the use of meta-learning for agricultural contexts in data-sparse regions and introduce task-informed meta-learning (TIML), an augmentation to model-agnostic meta-learning which takes advantage of task-specific metadata. We apply TIML to crop type classification and yield estimation, and find that TIML significantly improves performance compared to a range of benchmarks in both contexts, across a diversity of model architectures. While we focus on tasks from agriculture, TIML could offer benefits to any meta-learning setup with task-specific metadata, such as classification of geo-tagged images and species distribution modelling.

2022-02-03

ArXiv (preprint)

arxiv.org

Deep ReLU Networks Preserve Expected Length

Boris Hanin

Ryan Jeong

Assessing the complexity of functions computed by a neural network helps us understand how the network will learn and generalize. One natura… (see more)l measure of complexity is how the network distorts length - if the network takes a unit-length curve as input, what is the length of the resulting curve of outputs? It has been widely believed that this length grows exponentially in network depth. We prove that in fact this is not the case: the expected length distortion does not grow with depth, and indeed shrinks slightly, for ReLU networks with standard random initialization. We also generalize this result by proving upper bounds both for higher moments of the length distortion and for the distortion of higher-dimensional volumes. These theoretical results are corroborated by our experiments.

2022-01-27

ICLR.cc/2022/Conference (poster)

Generating physically-consistent high-resolution climate data with hard-constrained neural networks

Paula Harder

Qidong Yang

Venkatesh Ramesh

Prasanna Sattegeri

Alex Hernández-García

Campbell Watson

D. Szwarcman

The availability of reliable, high-resolution climate and weather data is important to inform long-term decisions on climate adaptation and … (see more)mitigation and to guide rapid responses to extreme events. Forecasting models are limited by computational costs and therefore often can only make coarse resolution predictions. Statistical downscaling can provide an efficient method of upsampling low-resolution data. In this field, deep learning has been applied successfully, often us-ing image super-resolution methods from computer vision. Despite achieving visually compelling results in some cases, such models often violate conservation laws when predicting physical variables. In order to conserve important physical quantities, we develop methods that guarantee physical constraints are satisfied by a deep downscaling model while also increasing their performance according to traditional metrics. We introduce two ways of constraining the network: A renor-malization layer added to the end of the neural network and a successive approach that scales with increasing upsampling factors. We show the applicability of our methods across different popular architectures and upsampling factors using ERA5 reanalysis data.

2021-12-31

arXiv.org (preprint)

TIML: Task-Informed Meta-Learning for crop type mapping

Gabriel Tseng

Hannah Kerner

Labeled datasets for agriculture are extremely spatially imbalanced. When developing algorithms for data-sparse regions, a previously explor… (see more)ed approach is to use transfer learning from data-rich regions. While standard transfer learning approaches typically leverage only direct inputs and outputs, geospatial imagery and agricultural data is rich in metadata that can inform transfer learning algorithms, such as the spatial coordinates of data-points. We build on previous work exploring use of meta-learning to crop type mapping in data-sparse regions and introduce task-informed meta-learning (TIML), an augmentation to model-agnostic meta-learning which takes advantage of this metadata. We apply TIML to the CropHarvest dataset, a global dataset of agricultural class labels paired with remote sensing data. In addition, we introduce the concept of forgetfulness when training meta-learning models on many similar tasks to mitigate memorization of training tasks. We find that TIML significantly improves average performance across the CropHarvest evaluation tasks compared to a range of benchmark models, measured using AUC ROC and F1 scores.

2021-12-31

AAAI.org/2022/Workshop/AIAFS (published)

Understanding the Evolution of Linear Regions in Deep Reinforcement Learning

Setareh Cohan

Nam Hee Gordon Kim

Michiel van de Panne

Policies produced by deep reinforcement learning are typically characterised by their learning curves, but they remain poorly understood in … (see more)many other respects. ReLU-based policies result in a partitioning of the input space into piecewise linear regions. We seek to understand how observed region counts and their densities evolve during deep reinforcement learning using empirical results that span a range of continuous control tasks and policy network dimensions. Intuitively, we may expect that during training, the region density increases in the areas that are frequently visited by the policy, thereby affording fine-grained control. We use recent theoretical and empirical results for the linear regions induced by neural networks in supervised learning settings for grounding and comparison of our results. Empirically, we find that the region density increases only moderately throughout training, as measured along fixed trajectories coming from the final policy. However, the trajectories themselves also increase in length during training, and thus the region densities decrease as seen from the perspective of the current trajectory. Our findings suggest that the complexity of deep reinforcement learning policies does not principally emerge from a significant growth in the complexity of functions observed on-and-around trajectories of the policy.

2021-12-31

Advances in Neural Information Processing Systems 35 (NeurIPS 2022) (published)

Hidden Hypergraphs, Error-Correcting Codes, and Critical Learning in Hopfield Networks

Christopher Hillar

Tenzin Chan

Rachel Taubman

In 1943, McCulloch and Pitts introduced a discrete recurrent neural network as a model for computation in brains. The work inspired breakthr… (see more)oughs such as the first computer design and the theory of finite automata. We focus on learning in Hopfield networks, a special case with symmetric weights and fixed-point attractor dynamics. Specifically, we explore minimum energy flow (MEF) as a scalable convex objective for determining network parameters. We catalog various properties of MEF, such as biological plausibility, and then compare to classical approaches in the theory of learning. Trained Hopfield networks can perform unsupervised clustering and define novel error-correcting coding schemes. They also efficiently find hidden structures (cliques) in graph theory. We extend this known connection from graphs to hypergraphs and discover n-node networks with robust storage of 2Ω(n1−ϵ) memories for any ϵ>0. In the case of graphs, we also determine a critical ratio of training samples at which networks generalize completely.

2021-11-10

Entropy (published)

Techniques for Symbol Grounding with SATNet

Sever Topan

Xujie Si

Many experts argue that the future of artificial intelligence is limited by the field's ability to integrate symbolic logical reasoning into… (see more) deep learning architectures. The recently proposed differentiable MAXSAT solver, SATNet, was a breakthrough in its capacity to integrate with a traditional neural network and solve visual reasoning problems. For instance, it can learn the rules of Sudoku purely from image examples. Despite its success, SATNet was shown to succumb to a key challenge in neurosymbolic systems known as the Symbol Grounding Problem: the inability to map visual inputs to symbolic variables without explicit supervision ("label leakage"). In this work, we present a self-supervised pre-training pipeline that enables SATNet to overcome this limitation, thus broadening the class of problems that SATNet architectures can solve to include datasets where no intermediary labels are available at all. We demonstrate that our method allows SATNet to attain full accuracy even with a harder problem setup that prevents any label leakage. We additionally introduce a proofreading method that further improves the performance of SATNet architectures, beating the state-of-the-art on Visual Sudoku.

2021-11-08

NeurIPS.cc/2021/Conference (spotlight)