Publications

MAPL: Parameter-Efficient Adaptation of Unimodal Pre-Trained Models for Vision-Language Few-Shot Prompting
Oscar Mañas
Pau Rodríguez
Saba Ahmadi
Aida Nematzadeh
Yash Goyal
Aishwarya Agrawal
Large pre-trained models have proved to be remarkable zero- and (prompt-based) few-shot learners in unimodal vision and language tasks. We p… (see more)ropose MAPL, a simple and parameter-efficient method that reuses frozen pre-trained unimodal models and leverages their strong generalization capabilities in multimodal vision-language (VL) settings. MAPL learns a lightweight mapping between the representation spaces of unimodal models using aligned image-text data, and can generalize to unseen VL tasks from just a few in-context examples. The small number of trainable parameters makes MAPL effective at low-data and in-domain learning. Moreover, MAPL's modularity enables easy extension to other pre-trained models. Extensive experiments on several visual question answering and image captioning benchmarks show that MAPL achieves superior or competitive performance compared to similar methods while training orders of magnitude fewer parameters. MAPL can be trained in just a few hours using modest computational resources and public datasets. We release our code and pre-trained model weights at https://github.com/mair-lab/mapl.
2023-04-30
Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics (published)
doi.org
arxiv.org
MeshDiffusion: Score-Based Generative 3D Mesh Modeling
Zhen Liu
Yao Feng
Michael J. Black
Derek Nowrouzezahrai
Liam Paull
Weiyang Liu
We consider the task of generating realistic 3D shapes, which is useful for a variety of applications such as automatic scene generation and… (see more) physical simulation. Compared to other 3D representations like voxels and point clouds, meshes are more desirable in practice, because (1) they enable easy and arbitrary manipulation of shapes for relighting and simulation, and (2) they can fully leverage the power of modern graphics pipelines which are mostly optimized for meshes. Previous scalable methods for generating meshes typically rely on sub-optimal post-processing, and they tend to produce overly-smooth or noisy surfaces without fine-grained geometric details. To overcome these shortcomings, we take advantage of the graph structure of meshes and use a simple yet very effective generative modeling method to generate 3D meshes. Specifically, we represent meshes with deformable tetrahedral grids, and then train a diffusion model on this direct parametrization. We demonstrate the effectiveness of our model on multiple generative tasks.
2023-04-30
International Conference on Learning Representations (Accept: notable-top-25%)
doi.org
openreview.net
Molecular Geometry Pretraining with SE(3)-Invariant Denoising Distance Matching
Shengchao Liu
Hongyu Guo
Jian Tang
Molecular representation pretraining is critical in various applications for drug and material discovery due to the limited number of labele… (see more)d molecules, and most existing work focuses on pretraining on 2D molecular graphs. However, the power of pretraining on 3D geometric structures has been less explored. This is owing to the difficulty of finding a sufficient proxy task that can empower the pretraining to effectively extract essential features from the geometric structures. Motivated by the dynamic nature of 3D molecules, where the continuous motion of a molecule in the 3D Euclidean space forms a smooth potential energy surface, we propose GeoSSL, a 3D coordinate denoising pretraining framework to model such an energy landscape. Further by leveraging an SE(3)-invariant score matching method, we propose GeoSSL-DDM in which the coordinate denoising proxy task is effectively boiled down to denoising the pairwise atomic distances in a molecule. Our comprehensive experiments confirm the effectiveness and robustness of our proposed method.
2023-04-30
International Conference on Learning Representations (Accept: poster)
doi.org
openreview.net
Neural Networks Efficiently Learn Low-Dimensional Representations with SGD
Alireza Mousavi-Hosseini
Sejun Park
Manuela Girotti
Ioannis Mitliagkas
Murat A. Erdogdu
We study the problem of training a two-layer neural network (NN) of arbitrary width using stochastic gradient descent (SGD) where the input …
2023-04-30
International Conference on Learning Representations (Accept: notable-top-25%)
doi.org
openreview.net
OC-0290 Investigation of the feasibility of selenium-75 as a viable brachytherapy source
J. Reid
Jonathan Kalinowski
J. Munro
A. Armstrong
S. Enger
2023-04-30
Radiotherapy and Oncology (published)
doi.org
PD-0334 Techniques to optimize auto-segmentation of small OARs in pediatric patients undergoing CSI
J. Tsui
M. Popovic
O. Ates
C. Hua
J. Schneider
S. Skamene
C. Freeman
S. Enger
2023-04-30
Radiotherapy and Oncology (published)
doi.org
PD-0505 Monte Carlo simulated correction factors of a novel phantom for brachytherapy dosimetry audits
K. Chelminski
R. Abdulrahim
A. Dimitriadis
E. Granizo-Roman
Jonathan Kalinowski
S. Enger
G. Azangwe
J. Swamidas
2023-04-30
Radiotherapy and Oncology (published)
doi.org
PD-0586 Design and assembly of a non-invasive radiation detector to measure the AIF in dynamic PET.
Liam Carroll
Y. Daoud
S. Enger
2023-04-30
Radiotherapy and Oncology (published)
doi.org
PO-1632 deep learning-based automatic segmentation of rectal tumors in endoscopy images
A. Thibodeau-Antonacci
L. Weishaupt
Aurélie Garant
C. Miller
T. Vuong
P. Nicolaï
S. Enger
2023-04-30
Radiotherapy and Oncology (published)
doi.org
PO-2166 Commissioning of new rectal applicator using electronic brachytherapy source
Nada Tomic
L. Liang
A. Esmaelbeigi
Jonathan Kalinowski
S. Enger
T. Vuong
S. Devic
2023-04-30
Radiotherapy and Oncology (published)
doi.org
PO-2225 Characterization of the RBE of various photon radiation qualities on human cancer cell lines
N. Chabaytah
J. Babik
Jian Li
B. Behmand
T. Connell
M. Evans
R. Ruo
H. Bekerat
S. Enger
T. Vuong
2023-04-30
Radiotherapy and Oncology (published)
doi.org
Predictive Inference with Feature Conformal Prediction
Jiaye Teng
Chuan Wen
Dinghuai Zhang
Yoshua Bengio
Yang Gao
Yang Yuan
Conformal prediction is a distribution-free technique for establishing valid prediction intervals. Although conventionally people conduct co… (see more)nformal prediction in the output space, this is not the only possibility. In this paper, we propose feature conformal prediction, which extends the scope of conformal prediction to semantic feature spaces by leveraging the inductive bias of deep representation learning. From a theoretical perspective, we demonstrate that feature conformal prediction provably outperforms regular conformal prediction under mild assumptions. Our approach could be combined with not only vanilla conformal prediction, but also other adaptive conformal prediction methods. Apart from experiments on existing predictive inference benchmarks, we also demonstrate the state-of-the-art performance of the proposed methods on large-scale tasks such as ImageNet classification and Cityscapes image segmentation.The code is available at https://github.com/AlvinWen428/FeatureCP.
2023-04-30
International Conference on Learning Representations (Accept: poster)
doi.org
openreview.net