Portrait de Pierfrancesco Melucci n'est pas disponible

Pierfrancesco Melucci

Alumni
Site web
Google Scholar

Publications

Steering Speech-Language Models: Training-Free Task Specialization via Contrastive Activation Addition
Séverin Baroudi
Yanis Labrak
Pierfrancesco Melucci
Sergio Burdisso
Petr Motlicek
Hervé Bredin
Mirco Ravanelli
Ricard Marxer
Activation steering has proven effective for controlling the behavior of Large Language Models (LLMs) at inference time, but its application… (voir plus) to SpeechLLMs remains new, and training-free steering approaches for such models are still largely unexplored. We propose a training-free Contrastive Activation Addition (CAA) protocol that derives steering vectors for common speech tasks (e.g. transcription) in SpeechLLMs from a small number of labeled utterances. We showcase that adding these vectors in the representation space, at inference time, enforces better the targeted speech task. We further show that, when combined with prompting, these vectors yield to consistent improvement over prompting alone on most evaluated tasks such as Automatic Speech Recognition (ASR) or Emotion Recognition (ER), and transfer to out-of-domain data. We additionally demonstrate the usefulness of script-normalization directions to enforce the target script of a specific language.
2026-10-02
arXiv (prépublication)
doi.org
arxiv.org