Mirco Ravanelli

heidari.allan@mila.quebec

Biographie

Mirco Ravanelli est professeur adjoint à l'Université Concordia, professeur associé à l'Université de Montréal et membre associé de Mila – Institut québécois d’intelligence artificielle. Lauréat du prix Amazon Research 2022, il est expert en apprentissage profond et en IA conversationnelle, et a publié plus de 60 articles dans ces domaines. Il se concentre principalement sur les nouveaux algorithmes d'apprentissage profond, y compris l'apprentissage autosupervisé, continu, multimodal, coopératif et économe en énergie. Mirco Ravanelli a effectué son postdoctorat à Mila, sous la direction du professeur Yoshua Bengio. Il est notamment le fondateur et le chef de file de SpeechBrain, l'une des boîtes à outils en code source ouvert les plus largement adoptées dans le domaine du traitement de la parole et de l'IA conversationnelle.

Étudiants actuels

Allan Allan

Baccalauréat - Concordia

gaspard.botte@mila.quebec

Gaspard Botté

Stagiaire de recherche - Concordia

briac.cordelle@mila.quebec

Cordelle Briac

Maîtrise recherche - Concordia University

Leo Brodeur

Maîtrise recherche - Concordia

leobrod44@gmail.com

Matthieu Cervera

Collaborateur·rice de recherche

Superviseur⋅e principal⋅e :

matthieu.cervera@mila.quebec

Victor Cruz

Maîtrise recherche - Concordia

victor.cruz@mila.quebec

Luca Della Libera

Doctorat - Concordia

Co-superviseur⋅e :

luca.dellalibera@mila.quebec

Wagner Drew

Maîtrise recherche - Concordia

Co-superviseur⋅e :

Irina Rish

drew.wagner@mila.quebec

Gianfranco Dumoulin Bertucci

Maîtrise recherche - Concordia

gianfranco.bertucci@mila.quebec

nadine.el-mufti@mila.quebec

Nadine El-Mufti

Maîtrise recherche - Concordia

Maab Elrashid Ahmed Mohamed

Doctorat - Concordia

Co-superviseur⋅e :

maab-elrashid.ahmed-mohamed@mila.quebec

francesco.bonzi@mila.quebec

Bonzi Francesco

Doctorat - Concordia

salman.hussainali@mila.quebec

Salman Sami Hussain Ali

Maîtrise recherche - Concordia University

sidra.intikhab@mila.quebec

Sidra Intikhab

Stagiaire de recherche - Concordia

Lovenya Jain

Collaborateur·rice de recherche - Université Laval

Superviseur⋅e principal⋅e :

f20211732@pilani.bits-pilani.ac.in

Jihoon Jeong

Doctorat - Université Laval

Superviseur⋅e principal⋅e :

jihoon.jeong@mila.quebec

Haoyu Li

Maîtrise professionnelle - Concordia Univesity

lihaoyuwilson@gmail.com

Eleonora Mancini

Collaborateur·rice alumni - UdeM

Superviseur⋅e principal⋅e :

eleonora.mancini@mila.quebec

ricard.marxer@mila.quebec

Ricard Marxer

Collaborateur·rice de recherche - University of Toulon

Superviseur⋅e principal⋅e :

Pablo Piantanida

pierfrancesco.melucci@mila.quebec

Pierfrancesco Melucci

Stagiaire de recherche - Concordia

Doctorat - UdeM

hussein-mohamu.jama@mila.quebec

Doctorat - Concordia

pooneh.mousavi@mila.quebec

Doctorat - Concordia

Co-superviseur⋅e :

Laurent Charlin

firat.oncel@mila.quebec

artem.ploujnikov@mila.quebec

Peter Peter

Postdoctorat - McGill

Doctorat - UdeM

Raphael Switzer

Stagiaire de recherche - Concordia

raphael.switzer@mila.quebec

benjamin.l.van.niekerk@gmail.com

Benjamin Van Niekerk

Postdoctorat - Concordia

FocalCodec : donner l’ouïe et la parole aux LLM à débit ultra-faible

Billets de blogue

Visual of FocalCodec,nouvelle méthode pour compresser la parole sans sacrifier la qualité, en vue d'obtenir des LLM multimodaux plus efficaces.

23 janvier 2026

par

Luca Della Libera

Francesco Paissan

Cem Subakan

Mirco Ravanelli

Lire l'article

13 juin 2024

SpeechBrain 1.0 : rendre l’IA conversationnelle accessible à tout le monde

par

Mirco Ravanelli

Lire l'article

Introducing SpeechBrain: A general-purpose PyTorch speech processing toolkit

28 avril 2021

Voici SpeechBrain : Une boîte à outils polyvalente de traitement de la parole basée sur PyTorch

par

Mirco Ravanelli

Loren Lugosch

Lire l'article

Publications

FocalCodec: Low-Bitrate Speech Coding via Focal Modulation Networks

Large language models have revolutionized natural language processing through self-supervised pretraining on massive datasets. Inspired by t… (voir plus)his success, researchers have explored adapting these methods to speech by discretizing continuous audio into tokens using neural audio codecs. However, existing approaches face limitations, including high bitrates, the loss of either semantic or acoustic information, and the reliance on multi-codebook designs when trying to capture both, which increases architectural complexity for downstream tasks. To address these challenges, we introduce FocalCodec, an efficient low-bitrate codec based on focal modulation that utilizes a single binary codebook to compress speech between 0.16 and 0.65 kbps. FocalCodec delivers competitive performance in speech resynthesis and voice conversion at lower bitrates than the current state-of-the-art, while effectively handling multilingual speech and noisy environments. Evaluation on downstream tasks shows that FocalCodec successfully preserves sufficient semantic and acoustic information, while also being well-suited for generative modeling. Demo samples and code are available at https://lucadellalib.github.io/focalcodec-web/.

2025-12-02

Conference on Neural Information Processing Systems (Accept (poster))

openreview.net

From Speech to Sonography: Spectral Networks for Ultrasound Microstructure Classification

Ali K. Z. Tehrani

An Tang

Guy Cloutier

Iman Rafati

Bich Ngoc Nguyen

Quoc-Huy Trinh

Ivan Rosado-Mendez

Hassan Rivaz

The frequency dependence of backscattered radiofrequency (RF) signals produced by ultrasound scanners carries rich information related to th… (voir plus)e tissue microstructure (i.e., scatterer size, attenuation). This information can be sue to classify tissues based on microstructural changes associated to disease onset and progression. Conventional convolutional neural networks (CNNs) can learn this information directly from radio-frequency (RF) data, but they often struggle to achieve adequate frequency selectivity. This increases model complexity and convergence time, and limits generalization. To overcome these challenges, SincNet, originally developed for speech processing, was adapted to classify RF data based on differences in frequency properties. Rather than learning every filter coefficient, SincNet only learns each filter's low frequency and bandwidth, dramatically reducing the number of parameters and improving frequency resolution. For model interpretability, a Gradient-Weighted Filter Contribution is introduced, which highlights the importance of spectral bands. The approach was validated on three datasets: simulated data with different scatterer sizes, experimental phantom data, and in vivo data of rats which were fed a methionine and choline- deficient diet to develop liver steatosis, inflammation, and fibrosis. The modified SincNet consistently achieved the best results in material/tissue classifications.

2025-11-26

IEEE transactions on bio-medical engineering (publié)

Investigating the Effectiveness of Explainability Methods in Parkinson's Detection from Speech

Paolo Torroni

Speech impairments in Parkinson's disease (PD) provide significant early indicators for diagnosis. While models for speech-based PD detectio… (voir plus)n have shown strong performance, their interpretability remains underexplored. This study systematically evaluates several explainability methods to identify PD-specific speech features, aiming to support the development of accurate, interpretable models for clinical decision-making in PD diagnosis and monitoring. Our methodology involves (i) obtaining attributions and saliency maps using mainstream interpretability techniques, (ii) quantitatively evaluating the faithfulness of these maps and their combinations obtained via union and intersection through a range of established metrics, and (iii) assessing the information conveyed by the saliency maps for PD detection from an auxiliary classifier. Our results reveal that, while explanations are aligned with the classifier, they often fail to provide valuable information for domain experts.

2025-04-05

ICASSPW @ IEEE International Conference on Acoustics, Speech, and Signal Processing (publié)

LMAC-TD: Producing Time Domain Explanations for Audio Classifiers

Neural networks are typically black-boxes that remain opaque with regards to their decision mechanisms. Several works in the literature have… (voir plus) proposed post-hoc explanation methods to alleviate this issue. This paper proposes LMAC-TD, a post-hoc explanation method that trains a decoder to produce explanations directly in the time domain. This methodology builds upon the foundation of L-MAC, Listenable Maps for Audio Classifiers, a method that produces faithful and listenable explanations. We incorporate SepFormer, a popular transformer-based time-domain source separation architecture. We show through a user study that LMAC-TD significantly improves the audio quality of the produced explanations while not sacrificing from faithfulness.

2025-04-05

IEEE Conference on Acoustics, Speech and Signal Processing (publié)

What Are They Doing? Joint Audio-Speech Co-Reasoning

Yingzhi Wang

Pooneh Mousavi

Artem Ploujnikov

In audio and speech processing, tasks usually focus on either the audio or speech modality, even when both sounds and human speech are prese… (voir plus)nt in the same audio clip. Recent Auditory Large Language Models (ALLMs) have made it possible to process audio and speech simultaneously within a single model, leading to further considerations of joint audio-speech tasks. In this paper, we establish a novel benchmark to investigate how well ALLMs can perform joint audio-speech processing. Specifically, we introduce Joint Audio-Speech Co-Reasoning (JASCO), a novel task that unifies audio and speech processing, strictly requiring co-reasoning across both modalities. We also release a scene-reasoning dataset called "What Are They Doing". Additionally, we provide deeper insights into the models' behaviors by analyzing their dependence on each modality.

2025-04-05

ICASSP 2025 - 2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (publié)

Audio Prototypical Network For Controllable Music Recommendation

Traditional recommendation systems represent user preferences in dense representations obtained through black-box encoder models. While thes… (voir plus)e models often provide strong recommendation performance, they lack interpretability for users, leaving users unable to understand or control the system's modeling of their preferences. This limitation is especially challenging in music recommendation, where user preferences are highly personal and often evolve based on nuanced qualities like mood, genre, tempo, or instrumentation. In this paper, we propose an audio prototypical network for controllable music recommendation. This network expresses user preferences in terms of prototypes representative of semantically meaningful features pertaining to musical qualities. We show that the model obtains competitive recommendation performance compared to popular baseline models while also providing interpretable and controllable user profiles.

2024-12-31

MLSP (publié)

Giuseppe Alessio D'Inverno

openreview.net

Generalization Limits of Graph Neural Networks in Identity Effects Learning

Simone Brugiapaglia

Graph Neural Networks (GNNs) have emerged as a powerful tool for data-driven learning on various graph domains. They are usually based on a … (voir plus)message-passing mechanism and have gained increasing popularity for their intuitive formulation, which is closely linked to the Weisfeiler-Lehman (WL) test for graph isomorphism to which they have been proven equivalent in terms of expressive power. In this work, we establish new generalization properties and fundamental limits of GNNs in the context of learning so-called identity effects, i.e., the task of determining whether an object is composed of two identical components or not. Our study is motivated by the need to understand the capabilities of GNNs when performing simple cognitive tasks, with potential applications in computational linguistics and chemistry. We analyze two case studies: (i) two-letters words, for which we show that GNNs trained via stochastic gradient descent are unable to generalize to unseen letters when utilizing orthogonal encodings like one-hot representations; (ii) dicyclic graphs, i.e., graphs composed of two cycles, for which we present positive existence results leveraging the connection between GNNs and the WL test. Our theoretical analysis is supported by an extensive numerical study.

2024-12-31

Neural Networks (publié)

A protocol for trustworthy EEG decoding with neural networks

Davide Borra

Elisa Magosso

2024-12-31

Neural Networks (publié)

Listenable Maps for Zero-Shot Audio Classifiers

Interpreting the decisions of deep learning models, including audio classifiers, is crucial for ensuring the transparency and trustworthines… (voir plus)s of this technology. In this paper, we introduce LMAC-ZS (Listenable Maps for Audio Classifiers in the Zero-Shot context), which, to the best of our knowledge, is the first decoder-based post-hoc interpretation method for explaining the decisions of zero-shot audio classifiers. The proposed method utilizes a novel loss function that maximizes the faithfulness to the original similarity between a given text-and-audio pair. We provide an extensive evaluation using the Contrastive Language-Audio Pretraining (CLAP) model to showcase that our interpreter remains faithful to the decisions in a zero-shot classification context. Moreover, we qualitatively show that our method produces meaningful explanations that correlate well with different text prompts.

2024-12-10

Conference on Neural Information Processing Systems (Accept (poster))

openreview.net

SpeechBrain-MOABB: An open-source Python library for benchmarking deep neural networks applied to EEG signals

Davide Borra

Francesco Paissan

2024-10-31

Computers in Biology and Medicine (publié)

Audio Editing with Non-Rigid Text Prompts

Francesco Paissan

Luca Della Libera

Zhepei Wang

Paris Smaragdis

In this paper, we explore audio-editing with non-rigid text edits. We show that the proposed editing pipeline is able to create audio edits … (voir plus)that remain faithful to the input audio. We explore text prompts that perform addition, style transfer, and in-painting. We quantitatively and qualitatively show that the edits are able to obtain results which outperform Audio-LDM, a recently released text-prompted audio generation model. Qualitative inspection of the results points out that the edits given by our approach remain more faithful to the input audio in terms of keeping the original onsets and offsets of the audio events.

2024-08-31

Interspeech (publié)

Speech Self-Supervised Representations Benchmarking: a Case for Larger Probing Heads

Salah Zaiem

Youcef Kemiche

Titouan Parcollet

Slim Essid

Self-supervised learning (SSL) leverages large datasets of unlabeled speech to reach impressive performance with reduced amounts of annotate… (voir plus)d data. The high number of proposed approaches fostered the emergence of comprehensive benchmarks that evaluate their performance on a set of downstream tasks exploring various aspects of the speech signal. However, while the number of considered tasks has been growing, most proposals rely upon a single downstream architecture that maps the frozen SSL representations to the task labels. This study examines how benchmarking results are affected by changes in the probing head architecture. Interestingly, we found that altering the downstream architecture structure leads to significant fluctuations in the performance ranking of the evaluated models. Against common practices in speech SSL benchmarking, we evaluate larger-capacity probing heads, showing their impact on performance, inference costs, generalization and multi-level feature exploitation.

2024-08-02

Computer Speech & Language (inconnu)