Portrait de Szu-wei Fu n'est pas disponible

Szu-wei Fu

Alumni

Publications

OSSEM: One-Shot Speaker Adaptive Speech Enhancement Using Meta Learning
Cheng Yu
Szu-wei Fu
Tsun-An Hsieh
Yu Tsao
Mirco Ravanelli
Although deep learning (DL) has achieved notable progress in speech enhancement (SE), further research is still required for a DL-based SE s… (voir plus)ystem to adapt effectively and efficiently to particular speakers. In this study, we propose a novel meta-learning-based speaker-adaptive SE approach (called OSSEM) that aims to achieve SE model adaptation in a one-shot manner. OSSEM consists of a modified transformer SE network and a speaker-specific masking (SSM) network. In practice, the SSM network takes an enrolled speaker embedding extracted using ECAPA-TDNN to adjust the input noisy feature through masking. To evaluate OSSEM, we designed a modified Voice Bank-DEMAND dataset, in which one utterance from the testing set was used for model adaptation, and the remaining utterances were used for testing the performance. Moreover, we set restrictions allowing the enhancement process to be conducted in real time, and thus designed OSSEM to be a causal SE system. Experimental results first show that OSSEM can effectively adapt a pretrained SE model to a particular speaker with only one utterance, thus yielding improved SE results. Meanwhile, OSSEM exhibits a competitive performance compared to state-of-the-art causal SE systems.
2022-09-17
Interspeech 2022 (publié)
doi.org
arxiv.org
MetricGAN+: An Improved Version of MetricGAN for Speech Enhancement
Szu-wei Fu
Cheng Yu
Tsun-An Hsieh
Peter Plantinga
Mirco Ravanelli
Xugang Lu
Yu Tsao
The discrepancy between the cost function used for training a speech enhancement model and human auditory perception usually makes the quali… (voir plus)ty of enhanced speech unsatisfactory. Objective evaluation metrics which consider human perception can hence serve as a bridge to reduce the gap. Our previously proposed MetricGAN was designed to optimize objective metrics by connecting the metric with a discriminator. Because only the scores of the target evaluation functions are needed during training, the metrics can even be non-differentiable. In this study, we propose a MetricGAN+ in which three training techniques incorporating domain-knowledge of speech processing are proposed. With these techniques, experimental results on the VoiceBank-DEMAND dataset show that MetricGAN+ can increase PESQ score by 0.3 compared to the previous MetricGAN and achieve state-of-the-art results (PESQ score = 3.15).
2021-12-31
Conference of the International Speech Communication Association (inconnu)
doi.org
arxiv.org
MetricGAN-U: Unsupervised Speech Enhancement/ Dereverberation Based Only on Noisy/ Reverberated Speech
Szu-wei Fu
Cheng Yu
Kuo-Hsuan Hung
Mirco Ravanelli
Yu Tsao
Most of the deep learning-based speech enhancement models are learned in a supervised manner, which implies that pairs of noisy and clean sp… (voir plus)eech are required during training. Consequently, several noisy speeches recorded in daily life cannot be used to train the model. Although certain unsupervised learning frameworks have also been proposed to solve the pair constraint, they still require clean speech or noise for training. Therefore, in this paper, we propose MetricGAN-U, which stands for MetricGAN-unsupervised, to further release the constraint from conventional unsupervised learning. In MetricGAN-U, only noisy speech is required to train the model by optimizing non-intrusive speech quality metrics. The experimental results verified that MetricGAN-U outperforms baselines in both objective and subjective metrics.
2021-12-31
IEEE International Conference on Acoustics, Speech and Signal Processing (inconnu)
doi.org
arxiv.org
SpeechBrain: A General-Purpose Speech Toolkit
Mirco Ravanelli
Titouan Parcollet
Peter Plantinga
Aku Rouhe
Samuele Cornell
Loren Lugosch
Cem Subakan
Nauman Dawalatabad
Abdelwahab Heba
Jianyuan Zhong
Ju-chieh Chou
Sung-Lin Yeh
Szu-wei Fu
Chien-Feng Liao
Elena Rastorgueva
François Grondin
William Aris
Hwidong Na
Yan Gao
Renato De Mori … (voir 1 de plus)
Yoshua Bengio
SpeechBrain is an open-source and all-in-one speech toolkit. It is designed to facilitate the research and development of neural speech proc… (voir plus)essing technologies by being simple, flexible, user-friendly, and well-documented. This paper describes the core architecture designed to support several tasks of common interest, allowing users to naturally conceive, compare and share novel speech processing pipelines. SpeechBrain achieves competitive or state-of-the-art performance in a wide range of speech benchmarks. It also provides training recipes, pretrained models, and inference scripts for popular speech datasets, as well as tutorials which allow anyone with basic Python proficiency to familiarize themselves with speech technologies.
2020-12-31
arXiv (prépublication)
doi.org
arxiv.org