Publications

Calm-Whisper: Reduce Whisper Hallucination On Non-Speech By Calming Crazy Heads Down
Yingzhi Wang
Anas Alhmoud
Saad Alsahly
Muhammad Alqurishi
Mirco Ravanaelli
2025-08-16
Interspeech 2025 (published)
doi.org
arxiv.org
LiSTEN: Learning Soft Token Embeddings for Neural Audio LLMs
Pooneh Mousavi
Shubham Gupta
Yusuf Cem Sübakan
Mirco Ravanaelli
Foundation models based on large language models (LLMs) have shown great success in handling various tasks and modalities. However, adapting… (see more) these models for general-purpose audio-language tasks is challenging due to differences in acoustic environments and task variations. In this work, we introduce LiSTEN Learning Soft Token Embeddings for Neural Audio LLMs), a framework for adapting LLMs to speech and audio tasks. LiSTEN uses a dynamic prompt selection strategy with learnable key-value pairs, allowing the model to balance general and task-specific knowledge while avoiding overfitting in a multitask setting. Our approach reduces dependence on large-scale ASR or captioning datasets, achieves competitive performance with fewer trainable parameters, and simplifies training by using a single-stage process. Additionally, LiSTEN enhances interpretability by analyzing the diversity and overlap of selected prompts across different tasks.
2025-08-16
Interspeech 2025 (published)
doi.org
arxiv.org
MuSACo: Multimodal Subject-Specific Selection and Adaptation for Expression Recognition with Co-Training
Muhammad Osama Zeeshan
Natacha Gillet
Alessandro Lameiras Koerich
Marco Pedersoli
Francois Bremond
Eric Granger
2025-08-16
ArXiv (preprint)
doi.org
arxiv.org
The NaijaVoices Dataset: Cultivating Large-Scale, High-Quality, Culturally-Rich Speech Data for African Languages
Chris Emezue
The NaijaVoices Community
Busayo Awobade
Abraham Toluwase Owodunni
Handel Emezue
Gloria Monica Tobechukwu Emezue
Nefertiti Nneoma Emezue
Sewade Ogun
Bunmi Akinremi
David Ifeoluwa Adelani
Chris Pal
2025-08-16
Interspeech (published)
doi.org
arxiv.org
SandboxSocial: A Sandbox for Social Media Using Multimodal AI Agents
Maximilian Puelma Touzel
Sneheel Sarangi
Gayatri Krishnakumar
Busra Tugce Gurbuz
Austin Welch
Zachary Yang
Andreea Musulan
Hao Yu
Ethan Kosak-Hine
Tom Gibbs
Camille Thibault
Reihaneh Rabbany
Jean-François Godbout
Dan Zhao
Kellin Pelrine
The online information ecosystem enables influence campaigns of unprecedented scale and impact. We urgently need empirically grounded approa… (see more)ches to counter the growing threat of malicious campaigns, now amplified by generative AI. But, developing defenses in real-world settings is impractical. Social system simulations with agents modelled using Large Language Models (LLMs) are a promising alternative approach and a growing area of research. However, existing simulators lack features needed to capture the complex information-sharing dynamics of platform-based social networks. To bridge this gap, we present SandboxSocial, a new simulator that includes several key innovations, mainly: (1) a virtual social media platform (modelled as Mastodon and mirrored in an actual Mastodon server) that enables a realistic setting in which agents interact; (2) an adapter that uses real-world user data to create more grounded agents and social media content; and (3) multi-modal capabilities that enable our agents to interact using both text and images---just as humans do on social media. We make the simulator more useful to researchers by providing measurement and analysis tools that track simulation dynamics and compute evaluation metrics to compare experimental results.
2025-08-15
Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence (published)
doi.org
Veracity: An Open-Source AI Fact-Checking System
Taylor Lynn Curtis
Maximilian Puelma Touzel
William Garneau
Manon Gruaz
Mike Pinder
Li Wei Wang
Sukanya Krishna
Luda Cohen
Jean-François Godbout
Reihaneh Rabbany
Kellin Pelrine
The proliferation of misinformation poses a significant threat to society, exacerbated by the capabilities of generative AI. This demo paper… (see more) introduces Veracity, an open-source AI system designed to empower individuals to combat misinformation through transparent and accessible fact-checking. Veracity leverages the synergy between Large Language Models (LLMs) and web retrieval agents to analyze user-submitted claims and provide grounded veracity assessments with intuitive explanations. Key features include multilingual support, numerical scoring of claim veracity, and an interactive interface inspired by familiar messaging applications. This paper will showcase Veracity's ability to not only detect misinformation but also explain its reasoning, fostering media literacy and promoting a more informed society.
2025-08-15
Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence (published)
doi.org
arxiv.org
Deep learning reveals that multidimensional social status drives population variation in 11,875 US participant cohort
Justin Marotta
Shambhavi Aggarwal
Nicole Osayande
Karin Saltoun
Jakub Kopal
Avram J. Holmes
Sarah W. Yip
Danilo Bzdok
As an increasing realization, many behavioral relationships are interwoven with inherent variations in human populations. Presently, there i… (see more)s no clarity in the biomedical community on which sources of population variation are most dominant. The recent advent of population-scale cohorts like the Adolescent Brain Cognitive DevelopmentSM Study (ABCD Study®) are now offering unprecedented depth and width of phenotype profiling that potentially explains interfamily differences. Here, we leveraged a deep learning framework (conditional variational autoencoder) on the totality of the ABCD Study® phenome (8,902 candidate phenotypes in 11,875 participants) to identify and characterize major sources of population stratification. 80% of the top 5 sources of explanatory stratifications were driven by distinct combinations of 202 available socioeconomic status (SES) measures; each in conjunction with a unique set of non-overlapping social and environmental factors. Several sources of variation across this cohort flagged geographies marked by material poverty interlocked with mental health and behavioral correlates. Deprivation emerged in another top stratification in relation to urbanicity and its ties to immigrant and racial and ethnic minoritized groups. Conversely, two other major sources of population variation were both driven by indicators of privilege: one highlighted measures of access to educational opportunity and income tied to healthy home environments and good behavior, the other profiled individuals of European ancestry leading advantaged lifestyles in desirable neighborhoods in terms of location and air quality. Overall, the disclosed social stratifications underscore the importance of treating SES as a multidimensional construct and recognizing its ties into social determinants of health.
2025-08-12
PLOS One (published)
doi.org
Deep learning reveals that multidimensional social status drives population variation in 11,875 US participant cohort
Justin Marotta
Shambhavi Aggarwal
Nicole Osayande
Karin Saltoun
Jakub Kopal
Avram J. Holmes
Sarah W. Yip
Danilo Bzdok
As an increasing realization, many behavioral relationships are interwoven with inherent variations in human populations. Presently, there i… (see more)s no clarity in the biomedical community on which sources of population variation are most dominant. The recent advent of population-scale cohorts like the Adolescent Brain Cognitive DevelopmentSM Study (ABCD Study®) are now offering unprecedented depth and width of phenotype profiling that potentially explains interfamily differences. Here, we leveraged a deep learning framework (conditional variational autoencoder) on the totality of the ABCD Study® phenome (8,902 candidate phenotypes in 11,875 participants) to identify and characterize major sources of population stratification. 80% of the top 5 sources of explanatory stratifications were driven by distinct combinations of 202 available socioeconomic status (SES) measures; each in conjunction with a unique set of non-overlapping social and environmental factors. Several sources of variation across this cohort flagged geographies marked by material poverty interlocked with mental health and behavioral correlates. Deprivation emerged in another top stratification in relation to urbanicity and its ties to immigrant and racial and ethnic minoritized groups. Conversely, two other major sources of population variation were both driven by indicators of privilege: one highlighted measures of access to educational opportunity and income tied to healthy home environments and good behavior, the other profiled individuals of European ancestry leading advantaged lifestyles in desirable neighborhoods in terms of location and air quality. Overall, the disclosed social stratifications underscore the importance of treating SES as a multidimensional construct and recognizing its ties into social determinants of health.
2025-08-12
Public Library of Science ONE (published)
doi.org
Pathfinding: a neurodynamical account of intuition
Steven Kotler
Michael Mannino
Karl Friston
Gyorgy Buzsáki
J. A. Scott Kelso
Guillaume Dumas
2025-08-12
Communications Biology (published)
doi.org
WeDesign: Generative AI-Facilitated Community Consultations for Urban Public Space Design
Rashid A. Mushkani
Hugo Berard
Shin Koseki
2025-08-12
ArXiv (preprint)
doi.org
arxiv.org
LIPTER, a cardiomyocyte-enriched long noncoding RNA, controls cardiac cytoskeletal maturation and is regulated by a cardiomyocyte-specific enhancer
Chukwuemeka George Anene-Nzelu
C.J Mick Lee
Svenja Koslowski
Wenhao Zheng
Marouane Benzaki
Michelle Mak
Xiaohua Wang
Lek Wen Tan
Albert Dashi
Yonglin Zhu
Talal Fawaz
Kenneth Ng
Davy Pham
Francis LeBlanc
Guillaume Lettre
Julie Hussin
Roger Foo
2025-08-11
BioRxiv (preprint)
doi.org
LIPTER, a cardiomyocyte-enriched long noncoding RNA, controls cardiac cytoskeletal maturation and is regulated by a cardiomyocyte-specific enhancer.
George Anene Nzelu
Mick Lee
Svenja Koslowski
Wenhao Zheng
Marouane Benzaki
Michelle Mak
Xiaohua Wang
Lek Wen Tan
Albert Dashi
Yonglin Zhu
Talal Fawaz
Kenneth Ng
Davy Pham
Francis LeBlanc
Guillaume Lettre
Julie Hussin
Roger Foo
Cardiac development is characterized by a complex series of molecular, cytoskeletal and electrophysiological changes that guarantee the prop… (see more)er functioning of adult cardiomyocytes (CMs). These changes are defined by cell-type-specific transcriptional rewiring of progenitor cells to form CMs, and are regulated by various epigenetic elements, such as long noncoding RNAs (lncRNAs). LncRNAs are versatile epigenetic regulators as they may act in cis or in trans to orchestrate important gene programs during cardiac development and may concurrently encode micropeptides. LIPTER is one such lncRNA, previously shown to regulate lipid droplet transport in cardiomyocytes and thus an important regulator of cardiomyocyte metabolism. Here we show that LIPTER also plays a role in the cytoskeletal maturation of CMs, as loss of LIPTER leads to persistent expression of fetal genes, changes in chromatin accessibility, disorganized sarcomeres and impaired calcium homeostasis in CMs. Furthermore, we have identified a cardiomyocyte-specific regulatory enhancer that regulates the expression of LIPTER in CMs. CRISPR-mediated inhibition of this enhancer led to reduced LIPTER expression in CMs and increased expression of fetal genes. This CM-specific enhancer could therefore be manipulated to control the expression of LIPTER for therapeutic benefit. In summary, we have unravelled a novel role of LIPTER in CMs cytoskeletal maturation and have identified a CM-specific enhancer for LIPTER.
2025-08-11
bioRxiv (preprint)
doi.org