2021 IEEE International Conference on Acoustics, Speech and Signal Processing
6-11 June 2021 • Toronto, Ontario, Canada
Extracting Knowledge from Information
| Paper ID | SPE-49.1 | ||
| Paper Title | CONTEXT-AWARE PROSODY CORRECTION FOR TEXT-BASED SPEECH EDITING | ||
| Authors | Max Morrison, Northwestern University, United States; Lucas Rencker, University of Surrey, United Kingdom; Zeyu Jin, Nicholas J. Bryan, Juan-Pablo Caceres, Bryan Pardo, Adobe Research, United States | ||
| Session | SPE-49: Speech Synthesis 7: General Topics | ||
| Location | Gather.Town | ||
| Session Time: | Friday, 11 June, 11:30 - 12:15 | ||
| Presentation Time: | Friday, 11 June, 11:30 - 12:15 | ||
| Presentation | Poster | ||
| Topic | Speech Processing: [SPE-SYNT] Speech Synthesis and Generation | ||
| IEEE Xplore Open Preview | Click here to view in IEEE Xplore | ||
| Abstract | Text-based speech editors expedite the process of editing speech recordings by permitting editing via intuitive cut, copy, and paste operations on a speech transcript. A major drawback of current systems, however, is that edited recordings often sound unnatural because of prosody mismatches around edited regions. In our work, we propose a new context-aware method for more natural sounding text-based editing of speech. To do so, we 1) use a series of neural networks to generate salient prosody features that are dependent on the prosody of speech surrounding the edit and amenable to fine-grained user control 2) use the generated features to control a standard pitch-shift and time-stretch method and 3) apply a denoising neural network to remove artifacts induced by the signal manipulation to yield a high-fidelity result. We evaluate our approach using a subjective listening test, provide a detailed comparative analysis, and conclude several interesting insights. | ||