Publications

Maxwell's Demon at Work: Efficient Pruning by Leveraging Saturation of Neurons
When training neural networks, dying neurons -- units becoming inactive or saturated -- are traditionally seen as harmful. This paper sheds … (see more)new light on this phenomenon. By exploring the impact of various hyperparameter configurations on dying neurons during training, we gather insights on how to improve upon sparse training approaches to pruning. We introduce Demon Pruning (DemP), a method that controls the proliferation of dead neurons through a combination of noise injection on active units and a one-cycle schedule regularization strategy, dynamically leading to network sparsity. Experiments on CIFAR-10 and ImageNet datasets demonstrate that DemP outperforms existing dense-to-sparse structured pruning methods, achieving better accuracy-sparsity tradeoffs and accelerating training by up to 3.56
Perception and neural representation of intermittent odor stimuli in mice
Luis Boero
Hao Wu
Joseph D. Zak
Farhad Pashakhanloo
Siddharth Jayakumar
Bahareh Tolooshams
Demba Ba
Venkatesh N. Murthy
scCobra allows contrastive cell embedding learning with domain adaptation for single cell data integration and harmonization
Bowen Zhao
Kailu Song
Dong-Qing Wei
Yi Xiong
Author Correction: Isospin competitions and valley polarized correlated insulators in twisted double bilayer graphene
Le Liu
Shihao Zhang
Yanbang Chu
Cheng Shen
Yuan Huang
Yalong Yuan
Jinpeng Tian
Yiru Ji
Rong Yang
Kenji Watanabe
Takashi Taniguchi
Dongxia Shi
Jianpeng Liu
Wei Yang
Guangyu Zhang
Modeling Multivariable High-resolution 3D Urban Microclimate Using Localized Fourier Neural Operator
Dongxue Zhan
Dingyang Geng
Wenhui Peng
Geng Tian
Yurong Shi
Naiping Gao
Xue Liu
Liangzhu (Leon) Wang
Accurate urban microclimate analysis with wind velocity and temperature is vital for energy-efficient urban planning, supporting carbon redu… (see more)ction, enhancing public health and comfort, and advancing the low-altitude economy. However, traditional computational fluid dynamics (CFD) simulations that couple velocity and temperature are computationally expensive. Recent machine learning advancements offer promising alternatives for accelerating urban microclimate simulations. The Fourier neural operator (FNO) has shown efficiency and accuracy in predicting single-variable velocity magnitudes in urban wind fields. Yet, for multivariable high-resolution 3D urban microclimate prediction, FNO faces three key limitations: blurry output quality, high GPU memory demand, and substantial data requirements. To address these issues, we propose a novel localized Fourier neural operator (Local-FNO) model that employs local training, geometry encoding, and patch overlapping. Local-FNO provides accurate predictions for rapidly changing turbulence in urban microclimate over 60 seconds, four times the average turbulence integral time scale, with an average error of 0.35 m/s in velocity and 0.30 °C in temperature. It also accurately captures turbulent heat flux represented by the velocity-temperature correlation. In a 2 km by 2 km domain, Local-FNO resolves turbulence patterns down to a 10 m resolution. It provides high-resolution predictions with 150 million feature dimensions on a single 32 GB GPU at nearly 50 times the speed of a CFD solver. Compared to FNO, Local-FNO achieves a 23.9% reduction in prediction error and a 47.3% improvement in turbulent fluctuation correlation.
HiPoNet: A Multi-View Simplicial Complex Network for High Dimensional Point-Cloud and Single-Cell Data
Hiren Madhu
Dhananjay Bhaskar
David R. Johnson
Rex Ying
Christopher Tape
Ian Adelstein
Michael Perlmutter
In this paper, we propose HiPoNet, an end-to-end differentiable neural network for regression, classification, and representation learning o… (see more)n high-dimensional point clouds. Our work is motivated by single-cell data which can have very high-dimensionality --exceeding the capabilities of existing methods for point clouds which are mostly tailored for 3D data. Moreover, modern single-cell and spatial experiments now yield entire cohorts of datasets (i.e., one data set for every patient), necessitating models that can process large, high-dimensional point-clouds at scale. Most current approaches build a single nearest-neighbor graph, discarding important geometric and topological information. In contrast, HiPoNet models the point-cloud as a set of higher-order simplicial complexes, with each particular complex being created using a reweighting of features. This method thus generates multiple constructs corresponding to different views of high-dimensional data, which in biology offers the possibility of disentangling distinct cellular processes. It then employs simplicial wavelet transforms to extract multiscale features, capturing both local and global topology from each view. We show that geometric and topological information is preserved in this framework both theoretically and empirically. We showcase the utility of HiPoNet on point-cloud level tasks, involving classification and regression of entire point-clouds in data cohorts. Experimentally, we find that HiPoNet outperforms other point-cloud and graph-based models on single-cell data. We also apply HiPoNet to spatial transcriptomics datasets using spatial coordinates as one of the views. Overall, HiPoNet offers a robust and scalable solution for high-dimensional data analysis.
Amortized In-Context Bayesian Posterior Estimation
N. L. Bracher
Priyank Jaini
Marcus Brubaker
Bayesian inference provides a natural way of incorporating prior beliefs and assigning a probability measure to the space of hypotheses. Cur… (see more)rent solutions rely on iterative routines like Markov Chain Monte Carlo (MCMC) sampling and Variational Inference (VI), which need to be re-run whenever new observations are available. Amortization, through conditional estimation, is a viable strategy to alleviate such difficulties and has been the guiding principle behind simulation-based inference, neural processes and in-context methods using pre-trained models. In this work, we conduct a thorough comparative analysis of amortized in-context Bayesian posterior estimation methods from the lens of different optimization objectives and architectural choices. Such methods train an amortized estimator to perform posterior parameter inference by conditioning on a set of data examples passed as context to a sequence model such as a transformer. In contrast to language models, we leverage permutation invariant architectures as the true posterior is invariant to the ordering of context examples. Our empirical study includes generalization to out-of-distribution tasks, cases where the assumed underlying model is misspecified, and transfer from simulated to real problems. Subsequently, it highlights the superiority of the reverse KL estimator for predictive problems, especially when combined with the transformer architecture and normalizing flows.
FairDropout: Using Example-Tied Dropout to Enhance Generalization of Minority Groups
Membership Inference Risks in Quantized Models: A Theoretical and Empirical Study
Quantizing machine learning models has demonstrated its effectiveness in lowering memory and inference costs while maintaining performance l… (see more)evels comparable to the original models. In this work, we investigate the impact of quantization procedures on the privacy of data-driven models, specifically focusing on their vulnerability to membership inference attacks. We derive an asymptotic theoretical analysis of Membership Inference Security (MIS), characterizing the privacy implications of quantized algorithm weights against the most powerful (and possibly unknown) attacks. Building on these theoretical insights, we propose a novel methodology to empirically assess and rank the privacy levels of various quantization procedures. Using synthetic datasets, we demonstrate the effectiveness of our approach in assessing the MIS of different quantizers. Furthermore, we explore the trade-off between privacy and performance using real-world data and models in the context of molecular modeling.
Overcoming Political Upheaval to Deliver Pediatric Surgical Care in Afghanistan: A Prospective Analysis of the First 1000 Procedures.
Dunya Moghul
Phillip J Hsu
Emma Bryce
Yalda Obaidy
Zane Hellman
Ajmal Sherzad
Maija Cheung
BACKGROUND Pediatric surgical care is limited in Afghanistan. Few details are known about the state of pediatric surgery in Afghanistan. We … (see more)explore the impact of a newly installed pediatric operating room by a children's charity on the provision of pediatric surgery in Afghanistan. STUDY DESIGN Following the opening in March 2023 of the new KidsOR operating room at Ataturk Hospital in Kabul, Afghanistan, perioperative clinical data was prospectively collected until December 2023. All children (age 14 years) undergoing surgical procedures were included in a REDCap database, and descriptive analyses were performed. RESULTS 1,014 opera
Using Image-based Al for insect monitoring and conservation - InsectAI COST Action
Tom August
Mario V Balzan
Paul Bodesheim
Gunnar Brehm
Lisette Cantú-Salazar
Sílvia Castro
Joseph Chipperfield
Guillaume Ghisbain
Alba Gomez-Segura
Jérémie Goulnik
Quentin Groom
Laurens Hogeweg
Chantal Huijbers
Andreas Kamilaris
Karolis Kazlauskis
Wouter Koch
Dimitri Korsch
João Loureiro
Youri Martin
Angeliki F Martinou … (see 27 more)
Kent McFarland
Xavier Mestdagh
Denis Michez
Charlie Outhwaite
Luca Pegoraro
Nadja Pernat
Lars B. Pettersson
Pavel Pipek
Cristina Preda
Tobias Roth
David B Roy
Helen Roy
Veljo Runnel
Martina Sasic
Dmitry Schigel
Julie Koch Sheard
Cecilie Svenningsen
Heliana Teixeira
Nicolas Titeux
Thomas Tscheulin
Elli Tzirkalli
Marijn van der Velde
Roel van Klink
Nicolas J Vereecken
Sarah Vray
Toke Thomas Høye
The InsectAI COST action will support insect monitoring and conservation at the national and continental scale in order to understand and co… (see more)unteract widespread insect declines. The Action will bring together a critical mass of researchers and stakeholders in image-based insect AI technologies to direct and drive the research agenda, build research capacity across Europe and support innovation and application. There is mounting evidence that populations of insects around the world are in sharp decline. Understanding trends in species and their drivers is key to knowing the size of the challenge, its causes and how to address it. To identify solutions that lead to sustainable biodiversity alongside economic prosperity, insect monitoring should be efficient and provide standardised and frequently updated status indicators to guide conservation actions. The EU Biodiversity Strategy 2030 identifies the critical challenge of delivering standardised information about the state of nature and image-based insect AI can contribute to this. Specifically, the EU Nature Restoration Law will likely set binding targets for the high resolution data that cameras can provide. Thus, outputs of the Action will contribute directly to EU policies implementation, where biodiversity monitoring is considered a key component. The InsectAI COST Action will organise workshops, conferences, short-term scientific missions, hackathons, design-sprints and much more, across four Working Groups. These groups will address how image-based insect AI technologies can best address Societal Needs, support innovation in Image Collection hardware, create standardised approaches for Image Processing and develop novel Data Analysis and Integration methods for turning data into actionable insights.
Improving Patient Safety Culture in Conflict-Affected Zones: A Cross-Sectional Survey of North Kivu Surgical Personnel in the Democratic Republic of the Congo.
Jacques Fadhili Bake
Claude Kasereka Masumbuko
Zacharie Tsongo Kibendelwa
Georges Bushu Lubuto
Jean‐Claude Mafuta Kyembwa
Esaie Kasereka Nzala
Papy Waleyirwe Kakule
Clovis Bwami Akumbi
Jean Zanga Kitutu
Tresor Basubi Wakilongo
Theophile Kubuya Hangi
Wilson Katembo Kwiraviwe
Benjamin Musemakweli
Beate Tshikudju Bahati
Steve Kisembo Bakabona
BACKGROUND Patient safety culture significantly impacts outcomes in surgery, where preventable errors can occur. This study assessed patient… (see more) safety culture and its determinants in operating rooms across North Kivu, a conflict-affected province in the eastern Democratic Republic of the Congo (DRC). METHODS A descriptive multicenter cross-sectional study was conducted from July to September 2024 in five urban and six rural hospitals. The French version of the Hospital Survey on Patient Safety Culture (HSOPSC) questionnaire was administered to 328 operating room healthcare professionals. RESULTS The response rate was 78% (256 completed surveys). Urban hospitals accounted for 55.5% of respondents, who were 73.4% male and 62.5% under the age of 40. The overall composite score for patient safety culture was 63.2%. Teamwork (81.1%) and management support for patient safety (77.7%) received the highest positive responses, whereas error reporting (39.9%) and patient safety event reporting (50%) scored lower. Half (49.6%) of the respondents rated patient safety as excellent or very good. There were no significant differences in overall mean composite scores between urban and rural hospitals (p = 0.677) and between medical and paramedical staff (p = 0.694). CONCLUSIONS The patient safety culture rating in North Kivu falls below international standards, highlighting an urgent need for improvement, particularly in error response and event reporting. Developing a tailored patient safety bundle for the region is essential to enhance overall health outcomes.