Relative Time Intervals Representation for Word-level Timestamping with Masked Training

The paper introduces a method for improving fine-grained, temporally aligned outputs in Speech Large Language Models (SpeechLLMs) by replacing absolute timestamps with relative timestamps, which reduces vocabulary size and enhances generalization. It proposes a hybrid fine‑tuning strategy that fully fine‑tunes the timestamp‑augmented embedding layer and language model head while applying LoRA to decoder layers, and introduces a masked timestamp training objective to prevent over‑reliance on ground‑truth timestamps. Experiments show significant gains in timestamp prediction accuracy without compromising transcription quality.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

• llms
• rag
• fine-tuning