Publications

CLIP-Mesh: Generating textured meshes from text using pretrained image-text models
Nasir Mohammad Khalid
Tianhao Xie
Eugene Belilovsky
Tiberiu Popa
We present a technique for zero-shot generation of a 3D model using only a target text prompt. Without any 3D supervision our method deforms… (voir plus) the control shape of a limit subdivided surface along with its texture map and normal map to obtain a 3D asset that corresponds to the input text prompt and can be easily deployed into games or modeling applications. We rely only on a pre-trained CLIP model that compares the input text prompt with differentiably rendered images of our 3D model. While previous works have focused on stylization or required training of generative models we perform optimization on mesh parameters directly to generate shape, texture or both. To constrain the optimization to produce plausible meshes and textures we introduce a number of techniques using image augmentations and the use of a pretrained prior that generates CLIP image embeddings given a text embedding.
2022-11-29
SIGGRAPH Asia 2022 Conference Papers (publié)
doi.org
arxiv.org
Visual Question Answering From Another Perspective: CLEVR Mental Rotation Tests
Christopher Beckham
Martin Weiss
Florian Golemo
Sina Honari
Derek Nowrouzezahrai
Christopher Pal
Different types of mental rotation tests have been used extensively in psychology to understand human visual reasoning and perception. Under… (voir plus)standing what an object or visual scene would look like from another viewpoint is a challenging problem that is made even harder if it must be performed from a single image. We explore a controlled setting whereby questions are posed about the properties of a scene if that scene was observed from another viewpoint. To do this we have created a new version of the CLEVR dataset that we call CLEVR Mental Rotation Tests (CLEVR-MRT). Using CLEVR-MRT we examine standard methods, show how they fall short, then explore novel neural architectures that involve inferring volumetric representations of a scene. These volumes can be manipulated via camera-conditioned transformations to answer the question. We examine the efficacy of different model variants through rigorous ablations and demonstrate the efficacy of volumetric representations.
2022-11-29
Pattern Recognition (inconnu)
doi.org
openreview.net
Histology-informed automatic parcellation of white matter tracts in the rat spinal cord
Harris Nami
Christian S. Perone
Julien Cohen-Adad
The white matter is organized into “tracts” or “bundles,” which connect different parts of the central nervous system. Knowing where… (voir plus) these tracts are located in each individual is important for understanding the cause of potential sensorial, motor or cognitive deficits and for developing appropriate treatments. Traditionally, tracts are found using tracer injection, which is a difficult, slow and poorly scalable technique. However, axon populations from a given tract exhibit specific characteristics in terms of morphometrics and myelination. Hence, the delineation of tracts could, in principle, be done based on their morphometry. The objective of this study was to generate automatic parcellation of the rat spinal white matter tracts using the manifold information from scanning electron microscopy images of the entire spinal cord. The axon morphometrics (axon density, axon diameter, myelin thickness and g-ratio) were computed pixelwise following automatic axon segmentation using AxonSeg. The parcellation was based on an agglomerative clustering algorithm to group the tracts. Results show that axon morphometrics provide sufficient information to automatically identify some white matter tracts in the spinal cord, however, not all tracts were correctly identified. Future developments of microstructure quantitative MRI even bring hope for a personalized clustering of white matter tracts in each individual patient. The generated atlas and the associated code can be found at https://github.com/neuropoly/tract-clustering.
2022-11-28
Frontiers in Neuroanatomy (publié)
doi.org
Improving the accuracy of single-trial fMRI response estimates using GLMsingle
Jacob S Prince
Ian Charest
Jan W Kurzawski
John A Pyles
Michael J Tarr
Kendrick Kay
2022-11-28
eLife (publié)
doi.org
Isometric Energies for Recovering Injectivity in Constrained Mapping
Xingyi Du
Danny M. Kaufman
Qingnan Zhou
Shahar Kovalsky
Yajie Yan
Noam Aigerman
Tao Ju
2022-11-28
ACM SIGGRAPH Conference and Exhibition on Computer Graphics and Interactive Techniques in Asia (publié)
doi.org
Score-based Denoising Diffusion with Non-Isotropic Gaussian Noise Models
Vikram Voleti
Christopher Pal
Adam Oberman
Generative models based on denoising diffusion techniques have led to an unprecedented increase in the quality and diversity of imagery that… (voir plus) is now possible to create with neural generative models. However, most contemporary state-of-the-art methods are derived from a standard isotropic Gaussian formulation. In this work we examine the situation where non-isotropic Gaussian distributions are used. We present the key mathematical derivations for creating denoising diffusion models using an underlying non-isotropic Gaussian noise model. We also provide initial experiments with the CIFAR10 dataset to help verify empirically that this more general modelling approach can also yield high-quality samples.
2022-11-28
NeurIPS.cc/2022/Workshop/SBM (poster)
doi.org
openreview.net
Continual Learning with Foundation Models: An Empirical Study of Latent Replay
Oleksiy Ostapenko
Timothee LESORT
Pau Rodríguez
Md Rifat Arefin
Arthur Douillard
Irina Rish
Laurent Charlin
Rapid development of large-scale pre-training has resulted in foundation models that can act as effective feature extractors on a variety of… (voir plus) downstream tasks and domains. Motivated by this, we study the efficacy of pre-trained vision models as a foundation for downstream continual learning (CL) scenarios. Our goal is twofold. First, we want to understand the compute-accuracy trade-off between CL in the raw-data space and in the latent space of pre-trained encoders. Second, we investigate how the characteristics of the encoder, the pre-training algorithm and data, as well as of the resulting latent space affect CL performance. For this, we compare the efficacy of various pre-trained models in large-scale benchmarking scenarios with a vanilla replay setting applied in the latent and in the raw-data space. Notably, this study shows how transfer, forgetting, task similarity and learning are dependent on the input data characteristics and not necessarily on the CL algorithms. First, we show that under some circumstances reasonable CL performance can readily be achieved with a non-parametric classifier at negligible compute. We then show how models pre-trained on broader data result in better performance for various replay sizes. We explain this with representational similarity and transfer properties of these representations. Finally, we show the effectiveness of self-supervised pre-training for downstream domains that are out-of-distribution as compared to the pre-training domain. We point out and validate several research directions that can further increase the efficacy of latent CL including representation ensembling. The diverse set of datasets used in this study can serve as a compute-efficient playground for further CL research. The codebase is available under https://github.com/oleksost/latent_CL.
2022-11-27
Proceedings of The 1st Conference on Lifelong Learning Agents (publié)
doi.org
proceedings.mlr.press
Improving Meta-Learning Generalization with Activation-Based Early-Stopping
Simon Guiroy
Christopher Pal
Goncalo Mordido
A. Chandar
2022-11-27
Proceedings of The 1st Conference on Lifelong Learning Agents (publié)
doi.org
proceedings.mlr.press
Pitfalls of Conditional Batch Normalization for Contextual Multi-Modal Learning
Ivaxi Sheth
Aamer Abdul Rahman
Mohammad Havaei
S Ebrahimi Kahou
Humans have perfected the art of learning from multiple modalities through sensory organs. Despite their impressive predictive performance o… (voir plus)n a single modality, neural networks cannot reach human level accuracy with respect to multiple modalities. This is a particularly challenging task due to variations in the structure of respective modalities. Conditional Batch Normalization (CBN) is a popular method that was proposed to learn contextual features to aid deep learning tasks. This technique uses auxiliary data to improve representational power by learning affine transformations for convolutional neural networks. Despite the boost in performance observed by using CBN layers, our work reveals that the visual features learned by introducing auxiliary data via CBN deteriorates. We perform comprehensive experiments to evaluate the brittleness of CBN networks to various datasets, suggesting that learning from visual features alone could often be superior for generalization. We evaluate CBN models on natural images for bird classification and histology images for cancer type classification. We observe that the CBN network learns close to no visual features on the bird classification dataset and partial visual features on the histology dataset. Our extensive experiments reveal that CBN may encourage shortcut learning between the auxiliary data and labels.
2022-11-27
ArXiv (prépublication)
doi.org
arxiv.org
Shimming toolbox: An open‐source software toolbox for <scp>B0</scp> and <scp>B1</scp> shimming in MRI
Alexandre D'Astous
Gaspard Cereza
Daniel Papp
Kyle M. Gilbert
Jason P. Stockmann
Eva Alonso‐Ortiz
Julien Cohen‐Adad
Introduce Shimming Toolbox ( https://shimming‐toolbox.org), an open‐source software package for prototyping new methods and performing s… (voir plus)tatic, dynamic, and real‐time B0 shimming as well as B1 shimming experiments. Shimming Toolbox features various field mapping techniques, manual and automatic masking for the brain and spinal cord, B0 and B1 shimming capabilities accessible through a user‐friendly graphical user interface. Validation of Shimming Toolbox was demonstrated in three scenarios: (i) B0 dynamic shimming in the brain at 7T using custom AC/DC coils, (ii) B0 real‐time shimming in the spinal cord at 3T, and (iii) B1 static shimming in the spinal cord at 7T. Shimming Toolbox provides an open‐source platform where researchers can collaborate, prototype and conveniently test B0 and B1 shimming experiments. Future versions will include additional field map preprocessing techniques, optimization algorithms, and compatibility across multiple MRI manufacturers.
2022-11-27
Magnetic Resonance in Medicine (inconnu)
doi.org
Receptive Field Refinement for Convolutional Neural Networks Reliably Improves Predictive Performance
Mats Leon Richter
Christopher Pal
Minimal changes to neural architectures (e.g. changing a single hyperparameter in a key layer), can lead to significant gains in predictive … (voir plus)performance in Convolutional Neural Networks (CNNs). In this work, we present a new approach to receptive field analysis that can yield these types of theoretical and empirical performance gains across twenty well-known CNN architectures examined in our experiments. By further developing and formalizing the analysis of receptive field expansion in convolutional neural networks, we can predict unproductive layers in an automated manner before ever training a model. This allows us to optimize the parameter-efficiency of a given architecture at low cost. Our method is computationally simple and can be done in an automated manner or even manually with minimal effort for most common architectures. We demonstrate the effectiveness of this approach by increasing parameter efficiency across past and current top-performing CNN-architectures. Specifically, our approach is able to improve ImageNet1K performance across a wide range of well-known, state-of-the-art (SOTA) model classes, including: VGG Nets, MobileNetV1, MobileNetV3, NASNet A (mobile), MnasNet, EfficientNet, and ConvNeXt - leading to a new SOTA result for each model class.
2022-11-25
ArXiv (prépublication)
doi.org
arxiv.org
Applied artificial intelligence in healthcare: Listening to the winds of change in a post-COVID-19 world
Arash Shaban-Nejad
Martin Michalowski
Simone Bianco
John S. Brownstein
David L Buckeridge
Robert L Davis
2022-11-24
Experimental Biology and Medicine (publié)
doi.org