Prosody-to-Text: Predicting text from low-pass filtered speech

The paper explores the reverse of the common prosody-from-text task by predicting text from low‑frequency speech features. Using a Whisper model fine‑tuned on only the 12 lowest Mel bins (≈450 Hz cutoff), the authors achieve a word error rate of 36 %, with 10 % of utterances perfectly recovered and 40 % below 25 % WER. They also report that, given the correct prefix, the next token is predicted correctly 79 % of the time, indicating a stronger link between low‑frequency prosody and lexical content than previously assumed.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

• llms
• fine-tuning