Publications

Bayesian Dynamic Causal Discovery
Alexander Tong
Lazar Atanackovic
Jason Hartford
Yoshua Bengio
Learning the causal structure of observable variables is a central focus for scientific discovery. Bayesian causal discovery methods tackle … (see more)this problem by learning a posterior over the set of admissible graphs that are equally likely given our priors and observations. Existing methods primarily consider observations from static systems and assume the underlying causal structure takes the form of a directed acyclic graph (DAG). In settings with dynamic feedback mechanisms that regulate the trajectories of individual variables, this acyclicity assumption fails unless we account for time. We treat causal discovery in the unrolled causal graph as a problem of sparse identification of a dynamical system. This imposes a natural temporal causal order between variables and captures cyclic feedback loops through time. Under this lens, we propose a new framework for Bayesian causal discovery for dynamical systems and present a novel generative flow network architecture (Dyn-GFN) tailored for this task. Dyn-GFN imposes an edge-wise sparse prior to sequentially build a k -sparse causal graph. Through evaluation on temporal data, our results show that the posterior learned with Dyn-GFN yields improved Bayes coverage of admissible causal structures relative to state of the art Bayesian causal discovery methods.
2022-11-30
NeurIPS.cc/2022/Workshop/CDS (poster)
openreview.net
openreview.net
CLIP-Mesh: Generating textured meshes from text using pretrained image-text models
Nasir M. Khalid
Tianhao Xie
Eugene Belilovsky
Tiberiu Popa
2022-11-30
SIGGRAPH Asia 2022 Conference Papers (published)
doi.org
arxiv.org
Histology-informed automatic parcellation of white matter tracts in the rat spinal cord
Harris Nami
Christian S. Perone
Julien Cohen-Adad
The white matter is organized into “tracts” or “bundles,” which connect different parts of the central nervous system. Knowing where… (see more) these tracts are located in each individual is important for understanding the cause of potential sensorial, motor or cognitive deficits and for developing appropriate treatments. Traditionally, tracts are found using tracer injection, which is a difficult, slow and poorly scalable technique. However, axon populations from a given tract exhibit specific characteristics in terms of morphometrics and myelination. Hence, the delineation of tracts could, in principle, be done based on their morphometry. The objective of this study was to generate automatic parcellation of the rat spinal white matter tracts using the manifold information from scanning electron microscopy images of the entire spinal cord. The axon morphometrics (axon density, axon diameter, myelin thickness and g-ratio) were computed pixelwise following automatic axon segmentation using AxonSeg. The parcellation was based on an agglomerative clustering algorithm to group the tracts. Results show that axon morphometrics provide sufficient information to automatically identify some white matter tracts in the spinal cord, however, not all tracts were correctly identified. Future developments of microstructure quantitative MRI even bring hope for a personalized clustering of white matter tracts in each individual patient. The generated atlas and the associated code can be found at https://github.com/neuropoly/tract-clustering.
2022-11-29
Frontiers in Neuroanatomy (published)
doi.org
Improving the accuracy of single-trial fMRI response estimates using GLMsingle
Jacob S Prince
Ian Charest
Jan W Kurzawski
John A Pyles
Michael J Tarr
Kendrick Kay
2022-11-29
eLife (published)
doi.org
Isometric Energies for Recovering Injectivity in Constrained Mapping
Xingyi Du
Danny M. Kaufman
Qingnan Zhou
Shahar Kovalsky
Yajie Yan
Noam Aigerman
Tao Ju
2022-11-29
ACM SIGGRAPH Conference and Exhibition on Computer Graphics and Interactive Techniques in Asia (published)
doi.org
Score-based Denoising Diffusion with Non-Isotropic Gaussian Noise Models
Vikram Voleti
Chris Pal
Adam M. Oberman
Generative models based on denoising diffusion techniques have led to an unprecedented increase in the quality and diversity of imagery that… (see more) is now possible to create with neural generative models. However, most contemporary state-of-the-art methods are derived from a standard isotropic Gaussian formulation. In this work we examine the situation where non-isotropic Gaussian distributions are used. We present the key mathematical derivations for creating denoising diffusion models using an underlying non-isotropic Gaussian noise model. We also provide initial experiments with the CIFAR10 dataset to help verify empirically that this more general modelling approach can also yield high-quality samples.
2022-11-29
NeurIPS.cc/2022/Workshop/SBM (poster)
doi.org
openreview.net
Continual Learning with Foundation Models: An Empirical Study of Latent Replay
Oleksiy Ostapenko
Timothee LESORT
Pau Rodriguez
Md Rifat Arefin
Arthur Douillard
Irina Rish
Laurent Charlin
Rapid development of large-scale pre-training has resulted in foundation models that can act as effective feature extractors on a variety of… (see more) downstream tasks and domains. Motivated by this, we study the efficacy of pre-trained vision models as a foundation for downstream continual learning (CL) scenarios. Our goal is twofold. First, we want to understand the compute-accuracy trade-off between CL in the raw-data space and in the latent space of pre-trained encoders. Second, we investigate how the characteristics of the encoder, the pre-training algorithm and data, as well as of the resulting latent space affect CL performance. For this, we compare the efficacy of various pre-trained models in large-scale benchmarking scenarios with a vanilla replay setting applied in the latent and in the raw-data space. Notably, this study shows how transfer, forgetting, task similarity and learning are dependent on the input data characteristics and not necessarily on the CL algorithms. First, we show that under some circumstances reasonable CL performance can readily be achieved with a non-parametric classifier at negligible compute. We then show how models pre-trained on broader data result in better performance for various replay sizes. We explain this with representational similarity and transfer properties of these representations. Finally, we show the effectiveness of self-supervised pre-training for downstream domains that are out-of-distribution as compared to the pre-training domain. We point out and validate several research directions that can further increase the efficacy of latent CL including representation ensembling. The diverse set of datasets used in this study can serve as a compute-efficient playground for further CL research. We will publish the code.
2022-11-28
Proceedings of The 1st Conference on Lifelong Learning Agents (published)
proceedings.mlr.press
arxiv.org
Improving Meta-Learning Generalization with Activation-Based Early-Stopping
Simon Guiroy
Chris Pal
Goncalo Mordido
Sarath Chandar
2022-11-28
Proceedings of The 1st Conference on Lifelong Learning Agents (published)
doi.org
arxiv.org
Shimming toolbox: An open‐source software toolbox for B0 and B1 shimming in MRI
Alexandre D'Astous
Gaspard Cereza
Daniel Papp
Kyle M. Gilbert
Jason P. Stockmann
Eva Alonso‐Ortiz
Julien Cohen-Adad
2022-11-28
Magnetic Resonance in Medicine (published)
doi.org
Shimming toolbox: An open‐source software toolbox for <scp>B0</scp> and <scp>B1</scp> shimming in MRI
Alexandre D'Astous
Gaspard Cereza
Daniel Papp
Kyle M. Gilbert
Jason P. Stockmann
Eva Alonso‐Ortiz
Julien Cohen-Adad
2022-11-28
Magnetic Resonance in Medicine (published)
doi.org
Receptive Field Refinement for Convolutional Neural Networks Reliably Improves Predictive Performance
Mats Leon Richter
Chris Pal
Minimal changes to neural architectures (e.g. changing a single hyperparameter in a key layer), can lead to significant gains in predictive … (see more)performance in Convolutional Neural Networks (CNNs). In this work, we present a new approach to receptive field analysis that can yield these types of theoretical and empirical performance gains across twenty well-known CNN architectures examined in our experiments. By further developing and formalizing the analysis of receptive field expansion in convolutional neural networks, we can predict unproductive layers in an automated manner before ever training a model. This allows us to optimize the parameter-efficiency of a given architecture at low cost. Our method is computationally simple and can be done in an automated manner or even manually with minimal effort for most common architectures. We demonstrate the effectiveness of this approach by increasing parameter efficiency across past and current top-performing CNN-architectures. Specifically, our approach is able to improve ImageNet1K performance across a wide range of well-known, state-of-the-art (SOTA) model classes, including: VGG Nets, MobileNetV1, MobileNetV3, NASNet A (mobile), MnasNet, EfficientNet, and ConvNeXt - leading to a new SOTA result for each model class.
2022-11-26
ArXiv (preprint)
doi.org
arxiv.org
Synergies between Disentanglement and Sparsity: Generalization and Identifiability in Multi-Task Learning
Sébastien Lachapelle
Tristan Deleu
Divyat Mahajan
Ioannis Mitliagkas
Yoshua Bengio
Simon Lacoste-Julien
Quentin Bertrand
Although disentangled representations are often said to be beneficial for downstream tasks, current empirical and theoretical understanding … (see more)is limited. In this work, we provide evidence that disentangled representations coupled with sparse base-predictors improve generalization. In the context of multi-task learning, we prove a new identifiability result that provides conditions under which maximally sparse base-predictors yield disentangled representations. Motivated by this theoretical result, we propose a practical approach to learn disentangled representations based on a sparsity-promoting bi-level optimization problem. Finally, we explore a meta-learning version of this algorithm based on group Lasso multiclass SVM base-predictors, for which we derive a tractable dual formulation. It obtains competitive results on standard few-shot classification benchmarks, while each task is using only a fraction of the learned representations.
2022-11-26
ArXiv (preprint)
arxiv.org
arxiv.org