Publications

The Singapore Consensus on Global AI Safety Research Priorities
Yoshua Bengio
Tegan Maharaj
Luke Ong
Stuart Russell
Dawn Song
Max Tegmark
Lan Xue
Ya-Qin Zhang
Stephen Casper
Wan Sie Lee
Sören Mindermann
Vanessa Wilfred
Vidhisha Balachandran
Fazl Barez
Michael Belinsky
Imane Bello
Malo Bourgon
Mark Brakel
Simeon Campos
Duncan Cass-Beggs … (see 68 more)
Jiahao Chen
Rumman Chowdhury
Kuan Chua Seah
Jeff Clune
Juntao Dai
Agnes Delaborde
Nouha Dziri
Francisco Eiras
Joshua Engels
Jinyu Fan
Adam Gleave
Noah Goodman
Fynn Heide
Johannes Heidecke
Dan Hendrycks
Cyrus Hodes
Bryan Low Kian Hsiang
Minlie Huang
Sami Jawhar
Wang Jingyu
Adam Tauman Kalai
Meindert Kamphuis
Mohan Kankanhalli
Subhash Kantamneni
Mathias Bonde Kirk
Thomas Kwa
Jeffrey Ladish
Kwok-Yan Lam
Wan Lee Sie
Taewhi Lee
Xiaojian Li
Jiajun Liu
Chaochao Lu
Yifan Mai
Richard Mallah
Julian Michael
Nick Moës
Simon Möller
Kihyuk Nam
Kwan Yee Ng
Mark Nitzberg
Besmira Nushi
Seán Ó hÉigeartaigh
Alejandro Ortega
Pierre Peigné
James Petrie
Benjamin Prud'homme
Reihaneh Rabbany
Nayat Sanchez-Pi
Sarah Schwettmann
Buck Shlegeris
Saad Siddiqui
Aradhana Sinha
Martín Soto
Cheston Tan
Dong Ting
William Tjhi
Robert Trager
Brian Tse
Anthony Tung K. H.
Vanessa Wilfred
John Willes
Denise Wong
Wei Xu
Rongwu Xu
Yi Zeng
HongJiang Zhang
Djordje Žikelić
Rapidly improving AI capabilities and autonomy hold significant promise of transformation, but are also driving vigorous debate on how to en… (see more)sure that AI is safe, i.e., trustworthy, reliable, and secure. Building a trusted ecosystem is therefore essential – it helps people embrace AI with confidence and gives maximal space for innovation while avoiding backlash. This requires policymakers, industry, researchers and the broader public to collectively work toward securing positive outcomes from AI’s development. AI safety research is a key dimension. Given that the state of science today for building trustworthy AI does not fully cover all risks, accelerated investment in research is required to keep pace with commercially driven growth in system capabilities. Goals: The 2025 Singapore Conference on AI (SCAI): International Scientific Exchange on AI Safety aims to support research in this important space by bringing together AI scientists across geographies to identify and synthesise research priorities in AI safety. The result, The Singapore Consensus on Global AI Safety Research Priorities, builds on the International AI Safety Report-A (IAISR) chaired by Yoshua Bengio and backed by 33 governments. By adopting a defence-in-depth model, this document organises AI safety research domains into three types: challenges with creating trustworthy AI systems (Development), challenges with evaluating their risks (Assessment), and challenges with monitoring and intervening after deployment (Control). Through the Singapore Consensus, we hope to globally facilitate meaningful conversations between AI scientists and AI policymakers for maximally beneficial outcomes. Our goal is to enable more impactful R&D efforts to rapidly develop safety and evaluation mechanisms and foster a trusted ecosystem where AI is harnessed for the public good.
2025-08-10
SuperIntelligence - Robotics - Safety & Alignment (published)
doi.org
arxiv.org
Untold stories: A qualitative investigation of patient and family experiences with congenital diaphragmatic hernia.
Alexandra Dimmer
Zanib Nafees
Sabrina Beauseigle
Franco A Carnevale
Elena Guadagno
Dan Poenaru
Pramod Puligandla
2025-08-10
Journal of Pediatric Surgery (published)
doi.org
In Which Areas of Technical AI Safety Could Geopolitical Rivals Cooperate?
BEN BUCKNALL
Saad Siddiqui
LARA THURNHERR
CONOR MCGURK
BEN HARACK
Anka Reuel
PATRICIA PASKOV
CASEY MAHONEY
Sören Mindermann
Scott Singer
VINAY HIREMATH
Charbel-Raphael Segerie
OSCAR DELANEY
Alessandro Abate
Fazl Barez
Michael K. Cohen
Philip Torr
FERENC HUSZÁR
ANISOARA CALINESCU
GABRIEL DAVIS JONES … (see 2 more)
Yoshua Bengio
Robert Trager
International cooperation is common in AI research, including between geopolitical rivals. While many experts advocate for greater internati… (see more)onal cooperation on AI safety to address shared global risks, some view cooperation on AI with suspicion, arguing that it can pose unacceptable risks to national security. However, the extent to which cooperation on AI safety poses such risks, as well as provides benefits, depends on the specific area of cooperation. In this paper, we consider technical factors that impact the risks of international cooperation on AI safety research, focusing on the degree to which such cooperation can advance dangerous capabilities, result in the sharing of sensitive information, or provide opportunities for harm. We begin by why nations historically cooperate on strategic technologies and analyse current US-China cooperation in AI as a case study. We further argue that existing frameworks for managing associated risks can be supplemented with consideration of key risks specific to cooperation on technical AI safety research. Through our analysis, we find that research into AI verification mechanisms and shared protocols may be suitable areas for such cooperation. Through this analysis we aim to help researchers and governments identify and mitigate the risks of international cooperation on AI safety research, so that the benefits of cooperation can be fully realised.
2025-08-10
SuperIntelligence - Robotics - Safety & Alignment (published)
doi.org
arxiv.org
In Which Areas of Technical AI Safety Could Geopolitical Rivals Cooperate?
BEN BUCKNALL
Saad Siddiqui
Robert Trager
Yoshua Bengio
LARA THURNHERR
CONOR MCGURK
BEN HARACK
International cooperation is common in AI research, including between geopolitical rivals. While many experts advocate for greater internati… (see more)onal cooperation on AI safety to address shared global risks, some view cooperation on AI with suspicion, arguing that it can pose unacceptable risks to national security. However, the extent to which cooperation on AI safety poses such risks, as well as provides benefits, depends on the specific area of cooperation. In this paper, we consider technical factors that impact the risks of international cooperation on AI safety research, focusing on the degree to which such cooperation can advance dangerous capabilities, result in the sharing of sensitive information, or provide opportunities for harm. We begin by why nations historically cooperate on strategic technologies and analyse current US-China cooperation in AI as a case study. We further argue that existing frameworks for managing associated risks can be supplemented with consideration of key risks specific to cooperation on technical AI safety research. Through our analysis, we find that research into AI verification mechanisms and shared protocols may be suitable areas for such cooperation. Through this analysis we aim to help researchers and governments identify and mitigate the risks of international cooperation on AI safety research, so that the benefits of cooperation can be fully realised.
2025-08-10
SuperIntelligence - Robotics - Safety & Alignment (published)
doi.org
An Empirical Study on Method-Level Performance Evolution in Open-Source Java Projects
Kaveh Shahedi
Nana Gyambrah
Heng Li
Maxime Lamothe
Foutse Khomh
Performance is a critical quality attribute in software development, yet the impact of method-level code changes on performance evolution re… (see more)mains poorly understood. While developers often make intuitive assumptions about which types of modifications are likely to cause performance regressions or improvements, these beliefs lack empirical validation at a fine-grained level. We conducted a large-scale empirical study analyzing performance evolution in 15 mature open-source Java projects hosted on GitHub. Our analysis encompassed 739 commits containing 1,499 method-level code changes, using Java Microbenchmark Harness (JMH) for precise performance measurement and rigorous statistical analysis to quantify both the significance and magnitude of performance variations. We employed bytecode instrumentation to capture method-specific execution metrics and systematically analyzed four key aspects: temporal performance patterns, code change type correlations, developer and complexity factors, and domain-size interactions. Our findings reveal that 32.7% of method-level changes result in measurable performance impacts, with regressions occurring 1.3 times more frequently than improvements. Contrary to conventional wisdom, we found no significant differences in performance impact distributions across code change categories, challenging risk-stratified development strategies. Algorithmic changes demonstrate the highest improvement potential but carry substantial regression risk. Senior developers produce more stable changes with fewer extreme variations, while code complexity correlates with increased regression likelihood. Domain-size interactions reveal significant patterns, with web server + small projects exhibiting the highest performance instability. Our study provides empirical evidence for integrating automated performance testing into continuous integration pipelines.
2025-08-08
ArXiv (preprint)
doi.org
arxiv.org
Low-Rank Expert Merging for Multi-Source Domain Adaptation in Person Re-Identification
Taha Mustapha Nehdi
Nairouz Mrabah
Atif Belal
Marco Pedersoli
Éric Granger
2025-08-08
ArXiv (preprint)
doi.org
arxiv.org
Spatio-Temporal Conditional Diffusion Models for Forecasting Future Multiple Sclerosis Lesion Masks Conditioned on Treatments
Gian Mario Favero
Ge Ya Luo
Nima Fathi
Justin Szeto
Douglas Arnold
Brennan Nichyporuk
Christopher Pal
Tal Arbel
2025-08-08
ArXiv (preprint)
doi.org
arxiv.org
CISO: Species Distribution Modeling Conditioned on Incomplete Species Observations
Hager Radi Abdelwahed
Mélisande Teng
Robin Zbinden
Laura Pollock
Hugo Larochelle
Devis Tuia
David Rolnick
Species distribution models (SDMs) are widely used to predict species' geographic distributions, serving as critical tools for ecological re… (see more)search and conservation planning. Typically, SDMs relate species occurrences to environmental variables representing abiotic factors, such as temperature, precipitation, and soil properties. However, species distributions are also strongly influenced by biotic interactions with other species, which are often overlooked. While some methods partially address this limitation by incorporating biotic interactions, they often assume symmetrical pairwise relationships between species and require consistent co-occurrence data. In practice, species observations are sparse, and the availability of information about the presence or absence of other species varies significantly across locations. To address these challenges, we propose CISO, a deep learning-based method for species distribution modeling Conditioned on Incomplete Species Observations. CISO enables predictions to be conditioned on a flexible number of species observations alongside environmental variables, accommodating the variability and incompleteness of available biotic data. We demonstrate our approach using three datasets representing different species groups: sPlotOpen for plants, SatBird for birds, and a new dataset, SatButterfly, for butterflies. Our results show that including partial biotic information improves predictive performance on spatially separate test sets. When conditioned on a subset of species within the same dataset, CISO outperforms alternative methods in predicting the distribution of the remaining species. Furthermore, we show that combining observations from multiple datasets can improve performance. CISO is a promising ecological tool, capable of incorporating incomplete biotic information and identifying potential interactions between species from disparate taxa.
2025-08-07
ArXiv (preprint)
doi.org
arxiv.org
Oligodendrogenesis Inhibition in the Juvenile and Adolescent Periods Differentially Alters Myelin in Mice
Lisa M. Gazdzinski
Jordan Mak
Kosaran Gumarathas
Miranda Mellerup
A. Collin
John G. Sled
Brian J. Nieman
Anne L. Wheeler
INTRODUCTION: The timing of myelination during development varies spatially according to the evolving functional demands of the maturing bra… (see more)in and is likely a mechanism of plasticity that contributes to sensitive periods of brain development during which the brain has heightened susceptibility to environmental influences. Disruption to this myelination process is therefore likely to have spatially and temporally heterogeneous effects. Myelinating oligodendrocytes arise from the differentiation of oligodendrocyte precursor cells, a process that depends on the transcription factor Myrf. In this study, the inducible Myrf conditional knockout mouse model is leveraged to characterize the impact of inhibiting oligodendrogenesis during the juvenile or adolescent period on white matter tracts with different timing of maturation. METHODS: Electron microscopy (EM) was used to quantify the fraction of myelinated axons, axon diameter, and myelin thickness, or T2- and diffusion-weighted MRI (dMRI) were used to compute white matter volumes and measures sensitive to microstructure. RESULTS: Mice with inhibited oligodendrogenesis during the juvenile period had a lower fraction of myelinated axons in the corpus callosum, which was not the case when oligodendrogenesis was halted during adolescence. Halting oligodendrogenesis in either developmental period had no effect on myelinated fraction in the earlier-to-mature optic tracts. Halted oligodendrogenesis during the juvenile period was detected with MRI as decreased volume of late-myelinating structures (corpus callosum, anterior commissure, and fornix) relative to controls. No group differences were observed in dMRI measures. Additionally, thinner myelin on larger calibre axons in the optics tracts of adolescent mice with halted oligodendrogenesis was detected with EM, but no MRI measures were sensitive to this difference. CONCLUSION: This study demonstrates that the impact of disrupting developmental oligodendrogenesis on white matter differs depending on the timing of disruption relative to the developmental stage of the structure. The results also highlight that morphological measures from structural MRI have high sensitivity to disrupted developmental myelination of white matter tracts.
2025-08-07
Developmental Neuroscience (published)
doi.org
Personalized Feature Translation for Expression Recognition: An Efficient Source-Free Domain Adaptation Method
Masoumeh Sharafi
Soufiane Belharbi
Houssem Ben Salem
Ali Etemad
Alessandro Lameiras Koerich
Marco Pedersoli
Simon Bacon
Éric Granger
Facial expression recognition (FER) models are employed in many video-based affective computing applications, such as human-computer interac… (see more)tion and healthcare monitoring. However, deep FER models often struggle with subtle expressions and high inter-subject variability, limiting their performance in real-world applications. To improve their performance, source-free domain adaptation (SFDA) methods have been proposed to personalize a pretrained source model using only unlabeled target domain data, thereby avoiding data privacy, storage, and transmission constraints. This paper addresses a challenging scenario where source data is unavailable for adaptation, and only unlabeled target data consisting solely of neutral expressions is available. SFDA methods are not typically designed to adapt using target data from only a single class. Further, using models to generate facial images with non-neutral expressions can be unstable and computationally intensive. In this paper, personalized feature translation (PFT) is proposed for SFDA. Unlike current image translation methods for SFDA, our lightweight method operates in the latent space. We first pre-train the translator on the source domain data to transform the subject-specific style features from one source subject into another. Expression information is preserved by optimizing a combination of expression consistency and style-aware objectives. Then, the translator is adapted on neutral target data, without using source data or image synthesis. By translating in the latent space, PFT avoids the complexity and noise of face expression generation, producing discriminative embeddings optimized for classification. Using PFT eliminates the need for image synthesis, reduces computational overhead (using a lightweight translator), and only adapts part of the model, making the method efficient compared to image-based translation.
2025-08-07
ArXiv (preprint)
doi.org
arxiv.org
Bias-inducing geometries: an exactly solvable data model with fairness implications
Stefano Sarao Mannelli
Federica Gerace
Negar Rostamzadeh
Luca Saglietti
2025-08-06
Physical Review E (published)
doi.org
arxiv.org
Revealing dynamic temporal trajectories and underlying regulatory networks with
<i>Cflows</i>
Alexander Tong
Manik Kuchroo
Shabarni Gupta
Aarthi Venkat
Chen Liu
Beatriz P. San Juan
Laura Rangel
Brandon Zhu
John G. Lock
Christine L. Chaffer
Smita Krishnaswamy
While single-cell technologies provide snapshots of tumor states, building continuous trajectories and uncovering causative gene regulatory … (see more)networks remains a significant challenge. We present Cflows , an AI framework that combines neural ODE networks with Granger causality to infer continuous cell state transitions and gene regulatory interactions from static scRNA-seq data. In a new 5-time point dataset capturing tumorsphere development over 30 days, Cflows reconstructs two types of trajectories leading to tumorsphere formation or apoptosis. Trajectory-based cell-of-origin analysis delineated a novel cancer stem cell profile characterized by CD44 hi EPCAM + CAV1 + , and uncovered a cell cycle–dependent enrichment of tumorsphere-initiating potential in G2/M or S-phase cells. Cflows uncovers ESRRA as a crucial causal driver of the tumor-forming gene regulatory network. Indeed, ESRRA inhibition significantly reduces tumor growth and metastasis in vivo. Cflows offers a powerful framework for uncovering cellular transitions and dynamic regulatory networks from static single-cell data.
2025-08-06
bioRxiv (preprint)
doi.org