Portrait of David Rolnick

David Rolnick

Core Academic Member
Canada CIFAR AI Chair
Assistant Professor, McGill University, School of Computer Science
Adjunct Professor, Université de Montréal, Department of Computer Science and Operations Research
Research Topics
Machine Learning Theory

Biography

David Rolnick is an assistant professor at McGill University’s School of Computer Science, a core academic member of Mila – Quebec Artificial Intelligence Institute and holds a Canada CIFAR AI Chair. Rolnick’s work focuses on applications of machine learning to help address climate change. He is the co-founder and chair of Climate Change AI, and scientific co-director of Sustainability in the Digital Age. After completing his PhD in applied mathematics at the Massachusetts Institute of Technology (MIT), he was a NSF Mathematical Sciences Postdoctoral Research Fellow, an NSF Graduate Research Fellow and a Fulbright Scholar. He was named to MIT Technology Review’s “35 Innovators Under 35” in 2021.

Current Students

Master's Research - McGill University
Collaborating Alumni - Université de Montréal
Collaborating researcher - The University of Dresden, Helmholtz Centre for Environmental Research Leipzig
Collaborating researcher
Collaborating researcher - National Observatory of Athens
Collaborating researcher
Collaborating researcher - McGill University
Collaborating researcher - KU Leuven
Master's Research - McGill University
Research Intern
Collaborating Alumni
Collaborating researcher - Université Paris-Saclay
Co-supervisor :
Collaborating researcher
Collaborating researcher
Postdoctorate - Université de Montréal
Principal supervisor :
Collaborating researcher - Université de Montréal
Research Intern - Johannes Kepler University
Master's Research - McGill University
PhD - McGill University
PhD - McGill University
Collaborating researcher - University of Waterloo
Collaborating researcher
Collaborating researcher
Research Intern - Université de Montréal
Postdoctorate - McGill University
Co-supervisor :
PhD - University of Waterloo
Co-supervisor :
PhD - Université de Montréal
Collaborating researcher
Master's Research - Osnabrueck university
Collaborating researcher
Co-supervisor :
PhD - McGill University
Collaborating researcher
Co-supervisor :
Collaborating researcher - Karlsruhe Institute of Technology
PhD - McGill University
Postdoctorate - Université de Montréal
Principal supervisor :
Collaborating researcher
Collaborating researcher - RWTH Aachen University (Rheinisch-Westfälische Technische Hochschule Aachen)
Co-supervisor :
PhD - McGill University
Postdoctorate - McGill University
Collaborating researcher
Collaborating researcher

Publications

Tree semantic segmentation from aerial image time series
Venkatesh Ramesh
Arthur Ouaknine
Stealing part of a production language model
Nicholas Carlini
Daniel Paleka
Krishnamurthy Dj Dvijotham
Thomas Steinke
Jonathan Hayase
A. Feder Cooper
Katherine Lee
Matthew Jagielski
Milad Nasr
Arthur Conmy
Eric Wallace
Florian Tramèr
We introduce the first model-stealing attack that extracts precise, nontrivial information from black-box production language models like … (see more)OpenAI's ChatGPT or Google's PaLM-2. Specifically, our attack recovers the embedding projection layer (up to symmetries) of a transformer model, given typical API access. For under \\
Improving Molecular Modeling with Geometric GNNs: an Empirical Study
Ali Ramlaoui
Théo Saulus
Basile Terver
Victor Schmidt
Fragkiskos D. Malliaros
Alexandre AGM Duval
Linear Weight Interpolation Leads to Transient Performance Gains
The Butterfly Effect: Tiny Perturbations Cause Neural Network Training to Diverge
Gül Sena Altıntaş
Devin Kwok
Neural network training begins with a chaotic phase in which the network is sensitive to small perturbations, such as those caused by stocha… (see more)stic gradient descent (SGD). This sensitivity can cause identically initialized networks to diverge both in parameter space and functional similarity. However, the exact degree to which networks are sensitive to perturbation, and the sensitivity of networks as they transition out of the chaotic phase, is unclear. To address this uncertainty, we apply a controlled perturbation at a single point in training time and measure its effect on otherwise identical training trajectories. We find that both the
Position: Application-Driven Innovation in Machine Learning
Alan Aspuru-Guzik
Sara Beery
Bistra Dilkina
Priya L. Donti
Marzyeh Ghassemi
Hannah Kerner
Claire Monteleoni
Esther Rolf
Milind Tambe
Adam White
Simultaneous linear connectivity of neural networks modulo permutation
Ekansh Sharma
Devin Kwok
Tom Denton
Daniel M. Roy
Application-Driven Innovation in Machine Learning
Alan Aspuru-Guzik
Sara Beery
Bistra N. Dilkina
Priya L. Donti
Marzyeh Ghassemi
Hannah Kerner
Claire Monteleoni
Esther Rolf
Milind Tambe
Adam White
As applications of machine learning proliferate, innovative algorithms inspired by specific real-world challenges have become increasingly i… (see more)mportant. Such work offers the potential for significant impact not merely in domains of application but also in machine learning itself. In this paper, we describe the paradigm of application-driven research in machine learning, contrasting it with the more standard paradigm of methods-driven research. We illustrate the benefits of application-driven machine learning and how this approach can productively synergize with methods-driven work. Despite these benefits, we find that reviewing, hiring, and teaching practices in machine learning often hold back application-driven innovation. We outline how these processes may be improved.
Predicting Species Occurrence Patterns from Partial Observations
Hager Radi
Mélisande Teng
To address the interlinked biodiversity and climate crises, we need an understanding of where species occur and how these patterns are chang… (see more)ing. However, observational data on most species remains very limited, and the amount of data available varies greatly between taxonomic groups. We introduce the problem of predicting species occurrence patterns given (a) satellite imagery, and (b) known information on the occurrence of other species. To evaluate algorithms on this task, we introduce SatButterfly, a dataset of satellite images, environmental data and observational data for butterflies, which is designed to pair with the existing SatBird dataset of bird observational data. To address this task, we propose a general model, R-Tran, for predicting species occurrence patterns that enables the use of partial observational data wherever found. We find that R-Tran outperforms other methods in predicting species encounter rates with partial information both within a taxon (birds) and across taxa (birds and butterflies). Our approach opens new perspectives to leveraging insights from species with abundant data to other species with scarce data, by modelling the ecosystems in which they co-occur.
Stealing Part of a Production Language Model
Nicholas Carlini
Daniel Paleka
Krishnamurthy Dvijotham
Thomas Steinke
Jonathan Hayase
A. Feder Cooper
Katherine Lee
Matthew Jagielski
Milad Nasr
Arthur Conmy
Eric Wallace
Florian Tramèr
Tackling Climate Change with Machine Learning: Fostering the Maturity of ML Applications for Climate Change
Shiva Madadkhani
Olivia Mendivil Ramos
Millie Chapman
Jesse Dunietz
Arthur Ouaknine
Dataset Difficulty and the Role of Inductive Bias
Devin Kwok
Nikhil Anand
Jonathan Frankle
Motivated by the goals of dataset pruning and defect identification, a growing body of methods have been developed to score individual examp… (see more)les within a dataset. These methods, which we call"example difficulty scores", are typically used to rank or categorize examples, but the consistency of rankings between different training runs, scoring methods, and model architectures is generally unknown. To determine how example rankings vary due to these random and controlled effects, we systematically compare different formulations of scores over a range of runs and model architectures. We find that scores largely share the following traits: they are noisy over individual runs of a model, strongly correlated with a single notion of difficulty, and reveal examples that range from being highly sensitive to insensitive to the inductive biases of certain model architectures. Drawing from statistical genetics, we develop a simple method for fingerprinting model architectures using a few sensitive examples. These findings guide practitioners in maximizing the consistency of their scores (e.g. by choosing appropriate scoring methods, number of runs, and subsets of examples), and establishes comprehensive baselines for evaluating scores in the future.