Publications

Pixels Under Pressure: Exploring Fine-Tuning Paradigms for Foundation Models in High-Resolution Medical Imaging
Zahra Tehrani Nasab
Amar Kumar
Tal Arbel
Advancements in diffusion-based foundation models have improved text-to-image generation, yet most efforts have been limited to low-resoluti… (voir plus)on settings. As high-resolution image synthesis becomes increasingly essential for various applications, particularly in medical imaging domains, fine-tuning emerges as a crucial mechanism for adapting these powerful pre-trained models to task-specific requirements and data distributions. In this work, we present a systematic study, examining the impact of various fine-tuning techniques on image generation quality when scaling to high resolution 512x512 pixels. We benchmark a diverse set of fine-tuning methods, including full fine-tuning strategies and parameter-efficient fine-tuning (PEFT). We dissect how different fine-tuning methods influence key quality metrics, including Fr\'echet Inception Distance (FID), Vendi score, and prompt-image alignment. We also evaluate the utility of generated images in a downstream classification task under data-scarce conditions, demonstrating that specific fine-tuning strategies improve both generation fidelity and downstream performance when synthetic images are used for classifier training and evaluation on real images. Our code is accessible through the project website - https://tehraninasab.github.io/PixelUPressure/.
2025-08-19
ArXiv (prépublication)
doi.org
arxiv.org
Field-level Comparison and Robustness Analysis of Cosmological <i>N</i>-body Simulations
Adrian E. Bayer
Francisco Villaescusa-Navarro
Sammy Sharief
Romain Teyssier
Lehman H. Garrison
Laurence Perreault-Levasseur
Greg L. Bryan
Marco Gatti
Eli Visbal
2025-08-18
The Astrophysical Journal (publié)
doi.org
Proceedings of the OHBM Open Science Room 2024
Selma Lugtmeijer
Ju-Chi Yu
Xiangzhen Kong
Lune Bellec
Janine D. Bijsterbosch
Elizabeth DuPre
Oscar Esteban
Ibrahim Faye
Seok-Jun Hong
Chuan-Peng Hu
Shella Keilholz
Chun-Chia Kung
Hyeong Hun Lee
Daniel Margulies
Cyril Pernet
Franco Pestilli
Jean-Baptiste Poline
Pradeep R. Raamana
Francesco Santini
Won Mok Shim … (voir 30 de plus)
Paul M. Thompson
Chao-Gan Yan
Niall W. Duncan
Nikhil Bhagwat
Peter Fox
Ana Van Gulick
David N. Kennedy
Gorana Pobric
Neda Sadeghi
Nick Souter
Sandeep Panta
Isabelle van der Velpen
Tonya White
Sina Mansour L.
Qing Wang
Povilas Karvelis
Anibal S. Heinsfeld
Yu-Fang Yang
Hong Ji Kim
Nur Shahidatul Nabila Binti Ibrahim
Stefano Moia
Wei Zhang
Jessica Haigh
Rose-Marie Kouwenhoven
Terra Hyun Lee
Hurshitha Vasudevan
Yuping Yang
Subapriya Suppiah
Yi-Ju Lee
Nils Muhlert
2025-08-18
Aperture Neuro (publié)
doi.org
MuSACo: Multimodal Subject-Specific Selection and Adaptation for Expression Recognition with Co-Training
Muhammad Osama Zeeshan
Natacha Gillet
Alessandro Lameiras Koerich
Marco Pedersoli
Francois Bremond
Eric Granger
Personalized expression recognition (ER) involves adapting a machine learning model to subject-specific data for improved recognition of exp… (voir plus)ressions with considerable interpersonal variability. Subject-specific ER can benefit significantly from multi-source domain adaptation (MSDA) methods, where each domain corresponds to a specific subject, to improve model accuracy and robustness. Despite promising results, state-of-the-art MSDA approaches often overlook multimodal information or blend sources into a single domain, limiting subject diversity and failing to explicitly capture unique subject-specific characteristics. To address these limitations, we introduce MuSACo, a multi-modal subject-specific selection and adaptation method for ER based on co-training. It leverages complementary information across multiple modalities and multiple source domains for subject-specific adaptation. This makes MuSACo particularly relevant for affective computing applications in digital health, such as patient-specific assessment for stress or pain, where subject-level nuances are crucial. MuSACo selects source subjects relevant to the target and generates pseudo-labels using the dominant modality for class-aware learning, in conjunction with a class-agnostic loss to learn from less confident target samples. Finally, source features from each modality are aligned, while only confident target features are combined. Our experimental results on challenging multimodal ER datasets: BioVid and StressID, show that MuSACo can outperform UDA (blending) and state-of-the-art MSDA methods.
2025-08-17
ArXiv (prépublication)
doi.org
arxiv.org
Rethinking Safety in LLM Fine-tuning: An Optimization Perspective
Minseon Kim
Jin Myung Kwak
Lama Alssum
Bernard Ghanem
Philip Torr
David Scott Krueger
Fazl Barez
Adel Bibi
Fine-tuning language models is commonly believed to inevitably harm their safety, i.e., refusing to respond to harmful user requests, even w… (voir plus)hen using harmless datasets, thus requiring additional safety measures. We challenge this belief through systematic testing, showing that poor optimization choices, rather than inherent trade-offs, often cause safety problems, measured as harmful responses to adversarial prompts. By properly selecting key training hyper-parameters, e.g., learning rate, batch size, and gradient steps, we reduce unsafe model responses from 16\% to approximately 5\%, as measured by keyword matching, while maintaining utility performance. Based on this observation, we propose a simple exponential moving average (EMA) momentum technique in parameter space that preserves safety performance by creating a stable optimization path and retains the original pre-trained model's safety properties. Our experiments on the Llama families across multiple datasets (Dolly, Alpaca, ORCA) demonstrate that safety problems during fine-tuning can largely be avoided without specialized interventions, outperforming existing approaches that require additional safety data while offering practical guidelines for maintaining both model performance and safety during adaptation.
2025-08-17
ArXiv (prépublication)
doi.org
arxiv.org
Increasing the Utility of Synthetic Images through Chamfer Guidance
Nicola Dall'Asen
Xiaofeng Zhang
Reyhane Askari Hemmat
Melissa Hall
Jakob Verbeek
Adriana Romero Soriano
Michal Drozdzal
Conditional image generative models hold considerable promise to produce infinite amounts of synthetic training data. Yet, recent progress i… (voir plus)n generation quality has come at the expense of generation diversity, limiting the utility of these models as a source of synthetic training data. Although guidance-based approaches have been introduced to improve the utility of generated data by focusing on quality or diversity, the (implicit or explicit) utility functions oftentimes disregard the potential distribution shift between synthetic and real data. In this work, we introduce Chamfer Guidance: a training-free guidance approach which leverages a handful of real exemplar images to characterize the quality and diversity of synthetic data. We show that by leveraging the proposed Chamfer Guidance, we can boost the diversity of the generations w.r.t. a dataset of real images while maintaining or improving the generation quality on ImageNet-1k and standard geo-diversity benchmarks. Our approach achieves state-of-the-art few-shot performance with as little as 2 exemplar real images, obtaining 96.4\% in terms of precision, and 86.4\% in terms of distributional coverage, which increase to 97.5\% and 92.7\%, respectively, when using 32 real images. We showcase the benefits of the Chamfer Guidance generation by training downstream image classifiers on synthetic data, achieving accuracy boost of up to 15\% for in-distribution over the baselines, and up to 16\% in out-of-distribution. Furthermore, our approach does not require using the unconditional model, and thus obtains a 31\% reduction in FLOPs w.r.t. classifier-free-guidance-based approaches at sampling time.
2025-08-14
ArXiv (prépublication)
doi.org
arxiv.org
Street Review: A Participatory AI-Based Framework for Assessing Streetscape Inclusivity
Rashid A. Mushkani
Shin (Alexandre) Koseki
2025-08-14
ArXiv (prépublication)
doi.org
arxiv.org
The Interpolation Constraint in the RV Analysis of M-Dwarfs Using Empirical Templates
Dhvani Doshi
Nicolas B. Cowan
'Etienne Artigau
René Doyon
Andr'e M. Silva
K. A. Moulla
Yashar Hezaveh
Precise radial velocity (pRV) measurements of M-dwarfs in the near-infrared (NIR) rely on empirical templates due to the lack of accurate st… (voir plus)ellar spectral models in this regime. Templates are assumed to approximate the true spectrum when constructed from many observations or in the high signal-to-noise limit. We develop a numerical simulation that generates SPIRou-like pRV observations from PHOENIX spectra, constructs empirical templates, and estimates radial velocities. This simulation solely considers photon noise and evaluates when empirical templates remain reliable for pRV analysis. Our results reveal a previously unrecognized noise source in templates, establishing a fundamental floor for template-based pRV measurements. We find that templates inherently include distortions in stellar line shapes due to imperfect interpolation at the detector's sampling resolution. The magnitude of this interpolation error depends on sampling resolution and RV content. Consequently, while stars with a higher RV content, such as cooler M-dwarfs are expected to yield lower RV uncertainties, their dense spectral features can amplify interpolation errors, potentially biasing RV estimates. For a typical M4V star, SPIRou's spectral and sampling resolution imposes an RV uncertainty floor of 0.5-0.8 m/s, independent of the star's magnitude or the telescope's aperture. These findings reveal a limitation of template-based pRV methods, underscoring the need for improved spectral modeling and better-than-Nyquist detector sampling to reach the next level of RV precision.
2025-08-14
ArXiv (prépublication)
arxiv.org
arxiv.org
Beyond Na\"ive Prompting: Strategies for Improved Zero-shot Context-aided Forecasting with LLMs
Arjun Ashok
Andrew Robert Williams
Vincent Zhihao Zheng
Irina Rish
Nicolas Chapados
Étienne Marcotte
Valentina Zantedeschi
Alexandre Drouin
Forecasting in real-world settings requires models to integrate not only historical data but also relevant contextual information, often ava… (voir plus)ilable in textual form. While recent work has shown that large language models (LLMs) can be effective context-aided forecasters via na\"ive direct prompting, their full potential remains underexplored. We address this gap with 4 strategies, providing new insights into the zero-shot capabilities of LLMs in this setting. ReDP improves interpretability by eliciting explicit reasoning traces, allowing us to assess the model's reasoning over the context independently from its forecast accuracy. CorDP leverages LLMs solely to refine existing forecasts with context, enhancing their applicability in real-world forecasting pipelines. IC-DP proposes embedding historical examples of context-aided forecasting tasks in the prompt, substantially improving accuracy even for the largest models. Finally, RouteDP optimizes resource efficiency by using LLMs to estimate task difficulty, and routing the most challenging tasks to larger models. Evaluated on different kinds of context-aided forecasting tasks from the CiK benchmark, our strategies demonstrate distinct benefits over na\"ive prompting across LLMs of different sizes and families. These results open the door to further simple yet effective improvements in LLM-based context-aided forecasting.
2025-08-13
ArXiv (prépublication)
doi.org
arxiv.org
Nested-ReFT: Efficient Reinforcement Learning for Large Language Model Fine-Tuning via Off-Policy Rollouts
Maxime Heuillet
Yufei CUI
Boxing Chen
Audrey Durand
Prasanna Parthasarathi
2025-08-13
ArXiv (prépublication)
doi.org
arxiv.org
Pathfinding: a neurodynamical account of intuition
Steven Kotler
Michael Mannino
Karl Friston
Gyorgy Buzsáki
J. A. Scott Kelso
Guillaume Dumas
2025-08-13
Communications Biology (publié)
doi.org
WeDesign: Generative AI-Facilitated Community Consultations for Urban Public Space Design
Rashid A. Mushkani
Hugo Berard
Shin (Alexandre) Koseki
Community consultations are integral to urban planning processes intended to incorporate diverse stakeholder perspectives. However, limited … (voir plus)resources, visual and spoken language barriers, and uneven power dynamics frequently constrain inclusive decision-making. This paper examines how generative text-to-image methods, specifically Stable Diffusion XL integrated into a custom platform (WeDesign), may support equitable consultations. A half-day workshop in Montreal involved five focus groups, each consisting of architects, urban designers, AI specialists, and residents from varied demographic groups. Additional data was gathered through semi-structured interviews with six urban planning professionals. Participants indicated that immediate visual outputs facilitated creativity and dialogue, yet noted issues in visualizing specific needs of marginalized groups, such as participants with reduced mobility, accurately depicting local architectural elements, and accommodating bilingual prompts. Participants recommended the development of an open-source platform incorporating in-painting tools, multilingual support, image voting functionalities, and preference indicators. The results indicate that generative AI can broaden participation and enable iterative interactions but requires structured facilitation approaches. The findings contribute to discussions on generative AI's role and limitations in participatory urban design.
2025-08-13
ArXiv (prépublication)
doi.org
arxiv.org