Neural Vocoders as Speech Enhancers

Speech enhancement (SE) and neural vocoding are traditionally viewed as separate tasks. In this work, we observe them under a common thread: the rank behavior of these processes. This observation prompts two key questions: $\mathit{\text{Can}}\mathit{\text{a}}\mathit{\text{model}}\mathit{\text{designed}}\mathit{\text{for}}\mathit{\text{one}}\mathit{\text{task’s}}\mathit{\text{rank}}\mathit{\text{degradation}}\mathit{\text{be}}\mathit{\text{adapted}}\mathit{\text{for}}\mathit{\text{the}}\mathit{\text{other?}}$ and $\mathit{\text{Is}}\mathit{\text{it}}\mathit{\text{possible}}\mathit{\text{to}}\mathit{\text{address}}\mathit{\text{both}}\mathit{\text{tasks}}\mathit{\text{using}}\mathit{\text{a}}\mathit{\text{unified}}\mathit{\text{model?}}$ Our empirical findings demonstrate that existing speech enhancement models can be successfully trained to perform vocoding tasks, and a single model, when jointly trained, can effectively handle both tasks with performance comparable to separately trained models. These results suggest that speech enhancement and neural vocoding can be unified under a broader framework of speech restoration. Code: https://github.com/Andong-Li-speech/Neural-Vocoders-as-Speech-Enhancers.