Publications

Scaling Language-Free Visual Representation Learning
David Fan
Shengbang Tong
Jiachen Zhu
Koustuv Sinha
Zhuang Liu
Xinlei Chen
Michael G. Rabbat
Nicolas Ballas
Yann Lecun
Amir Bar
Saining Xie
2025-03-31
ArXiv (preprint)
doi.org
arxiv.org
A Systematic Review of the Empirical Use of the CCHS-MH (Canadian Community Health Survey–Mental Health) Survey
Maria Cutumisu
2025-03-31
Proceedings of the 2025 AERA Annual Meeting (published)
doi.org
Trade‐off of different deep learning‐based auto‐segmentation approaches for treatment planning of pediatric craniospinal irradiation autocontouring of OARs for pediatric CSI
Alana Thibodeau‐Antonacci
Marija Popovic
Ozgur Ates
Chia‐Ho Hua
James Schneider
Sonia Skamene
Carolyn Freeman
S. Enger
James Man Git Tsui
As auto‐segmentation tools become integral to radiotherapy, more commercial products emerge. However, they may not always suit our needs. … (see more)One notable example is the use of adult‐trained commercial software for the contouring of organs at risk (OARs) of pediatric patients.
2025-03-31
Medical Physics (Lancaster) (published)
doi.org
NoProp: Training Neural Networks without Back-propagation or Forward-propagation
Qinyu Li
Yee Whye Teh
Razvan Pascanu
2025-03-30
ArXiv (preprint)
doi.org
arxiv.org
Universal algorithm for transforming Hamiltonian eigenvalues
Tatsuki Odake
Hlér Kristjánsson
Philip Taranto
Mio Murao
Manipulating Hamiltonians governing physical systems has found a broad range of applications, from quantum chemistry to semiconductor design… (see more). In this work, we provide a new way of manipulating Hamiltonians, by transforming their eigenvalues while keeping their eigenstates unchanged. We develop a universal algorithm that deterministically implements any desired (suitably differentiable) function on the eigenvalues of any unknown Hamiltonian, whose positive-time and negative-time dynamics are given as a black box. Our algorithm uses correlated randomness to efficiently combine two subroutines -- namely controlization and Fourier series simulation -- exemplifying a general compilation procedure that we develop. The time complexity of our algorithm is significantly reduced via said compilation technique compared to a na{ï}ve concatenation of the subroutines and outperforms similar methods based on the quantum singular value transformation.
2025-03-30
Physical Review Research (published)
doi.org
arxiv.org
Steering CLIP's vision transformer with sparse autoencoders
Sonia Joseph
Praneet Suresh
Ethan Goldfarb
Lorenz Hufe
Yossi Gandelsman
Robert Graham
Danilo Bzdok
Wojciech Samek
Blake Aaron Richards
While vision models are highly capable, their internal mechanisms remain poorly understood-- a challenge which sparse autoencoders (SAEs) ha… (see more)ve helped address in language, but which remains underexplored in vision. We address this gap by training SAEs on CLIP's vision transformer and uncover key differences between vision and language processing, including distinct sparsity patterns for SAEs trained across layers and token types. We then provide the first systematic analysis of the steerability of CLIP's vision transformer by introducing metrics to quantify how precisely SAE features can be steered to affect the model's output. We find that 10-15% of neurons and features are steerable, with SAEs providing thousands more steerable features than the base model. Through targeted suppression of SAE features, we then demonstrate improved performance on three vision disentanglement tasks (CelebA, Waterbirds, and typographic attacks), finding optimal disentanglement in middle model layers, and achieving state-of-the-art performance on defense against typographic attacks. We release our CLIP SAE models and code to support future research in vision transformer interpretability.
2025-03-29
MIV @ IEEE/CVF Conference on Computer Vision and Pattern Recognition (poster)
doi.org
openreview.net
Bridging biodiversity and ecosystem services through useful plant species
Nina Obiar
Isaac Eckert
Janelle Baker
Daniel Moerman
Laura J. Pollock
2025-03-27
Plants, People, Planet (published)
doi.org
Conditional Diffusion Models are Medical Image Classifiers that Provide Explainability and Uncertainty for Free
Gian Mario Favero
Parham Saremi
Emily Kaczmarek
Brennan Nichyporuk
Tal Arbel
2025-03-26
Medical Imaging with Deep Learning (oral)
doi.org
proceedings.mlr.press
debug-gym: A Text-Based Environment for Interactive Debugging
Xingdi Yuan
Morgane M Moss
Charbel Feghali
Chinmay Singh
Darya Moldavskaya
Drew MacPhee
Lucas Caccia
Matheus Pereira
Minseon Kim
Alessandro Sordoni
Marc-Alexandre Côté
2025-03-26
ArXiv (preprint)
doi.org
arxiv.org
How do language models learn facts? Dynamics, curricula and hallucinations
Nicolas Zucchet
Jörg Bornschein
Stephanie Chan
Andrew Lampinen
Razvan Pascanu
Soham De
2025-03-26
ArXiv (preprint)
doi.org
arxiv.org
PRISM: High-Resolution & Precise Counterfactual Medical Image Generation using Language-guided Stable Diffusion
Amar Kumar
Anita Kriz
Mohammad Havaei
Tal Arbel
Developing reliable and generalizable deep learning systems for medical imaging faces significant obstacles due to spurious correlations, da… (see more)ta imbalances, and limited text annotations in datasets. Addressing these challenges requires architectures robust to the unique complexities posed by medical imaging data. The rapid advancements in vision-language foundation models within the natural image domain prompt the question of how they can be adapted for medical imaging tasks. In this work, we present PRISM, a framework that leverages foundation models to generate high-resolution, language-guided medical image counterfactuals using Stable Diffusion. Our approach demonstrates unprecedented precision in selectively modifying spurious correlations (the medical devices) and disease features, enabling the removal and addition of specific attributes while preserving other image characteristics. Through extensive evaluation, we show how PRISM advances counterfactual generation and enables the development of more robust downstream classifiers for clinically deployable solutions. To facilitate broader adoption and research, we make our code publicly available at https://github.com/Amarkr1/PRISM.
2025-03-26
MIDL.io/2025/Conference (oral)
doi.org
openreview.net
StarFlow: Generating Structured Workflow Outputs From Sketch Images
Patrice Béchard
Chao Wang
Amirhossein Abaskohi
Juan A. Rodriguez
Christopher Pal
David Vázquez
Spandana Gella
Sai Rajeswar
Perouz Taslakian
2025-03-26
ArXiv (preprint)
doi.org
arxiv.org