Marimba: A System One Model for Speech
arXiv:2611.18330v1 Announce Type: cross Abstract: Speech applications require decisions about term presence, turn completion, and language within an audio window. We present Marimba, an interface for...
Earlybird is a 29‑million‑parameter streaming speech recognizer that predicts words before the speaker finishes them, achieving median correct predictions 90 ms early on clean audiobook speech. It learns from audio cut off at random points, including mid‑word, and outputs confidence scores for every word, including the one still being spoken. Across six diverse test sets—audible books, TED talks, public speech, earnings calls, and synthetic speech—Earlybird consistently outperforms existing streaming models, correctly predicting 69 % of words before they finish on audiobooks, compared to 3–25 % for other models.
arXiv:2611.18330v1 Announce Type: cross Abstract: Speech applications require decisions about term presence, turn completion, and language within an audio window. We present Marimba, an interface for...
The study evaluates conversational speech models by placing them at 427 pauses and 482 turn ends from 176 unscripted conversations, scoring their responses on timing, latency, length, wording, and recall of recent dialogue. Hosted APIs (GPT Realtime, Gemini 3.1 Flash Live, Grok Voice) respond at almost every pause, with latency ranging from 1–4 s for silence-based endpointers and about 8 s for semantic turn detection, while full‑duplex models (PersonaPlex, Moshi) remain silent more often and can interrupt within human reaction time. Recall rates vary: hosted systems remember 41–60 % of facts under a Claude Opus 5.5 judge and 19–26 % with a strict word matcher; full‑duplex models recall 17–22 % and 12–15 % respectively.
Antalia-2 Mini is a single‑voice Turkish text‑to‑speech model with 7.6 million parameters that reads characters and outputs 48 kHz audio at 41× real time on an Apple M5 laptop CPU. It was trained solely on synthetic speech from the larger VoxCPM2 teacher, and fine‑tuning its vocoder on the model’s own mel spectrograms reduced Whisper’s word error rate from 3.89 % to 1.75 %. On the 495‑sentence Freya‑TR‑Eval benchmark, version 1.0.1 achieves a 1.09 % WER and 0.22 % character error rate, comparable to the EMA Lightning model in full‑band UTMOS scores.
The paper introduces AWED, a word‑level emission‑delay metric, and demonstrates that post‑training a delayed‑streaming model with a joint reward (GRPO) improves both accuracy and latency. Using a single operating point (τ=6 frames), the method outperforms both its supervised baseline and the Voxtral Realtime backbone across all lookahead budgets, reducing WER by up to 30.8% at 80 ms delay and lowering median AWED from 1.17 s to 1.04 s.
arXiv:2609.38232v1 Announce Type: cross Abstract: Streaming spoken agents may take an external action before the available speech supports it, yet final-turn scores do not reveal whether each observe...
FA-Bench is an open framework that standardizes forced alignment evaluation by fixing transcript normalization, data splits, phone mapping, and scoring procedures. It offers two tracks: one with reference transcripts and another with recognizer outputs, evaluated on clean and four degraded audio conditions using 21 open models and 9 commercial APIs. The benchmark scores every utterance boundary with a tolerance‑based F1 metric, revealing systematic timing biases such as Whisper’s 150 ms early alignment and commercial APIs’ 50 ms late alignment.
The paper presents a taxonomy of errors in streaming dictation ASR systems, moving beyond traditional word error rate (WER) metrics. It evaluates eleven ASR systems on a diverse test set of 106 scripted English dictations, 20 clips each in eleven other languages/dialects, and 400 bilingual recordings, categorizing errors into recognition, output formatting, language handling, and audio drop‑out. The study finds that systems with similar WER can fail in markedly different ways—e.g., some miss a large proportion of whispered words or fail to capture non‑English content—highlighting the importance of error type analysis for real‑time dictation. whyItMatters":"The findings reveal that conventional WER alone is insufficient for assessing real‑time dictation performance, as it masks critical failure modes that directly affect user experience."
Alania-2 is a Turkish text‑to‑speech system featuring 488 M parameters and the VoxCPM architecture, trained from scratch on 18,195 hours of Turkish speech—90 % of which is synthetic. It offers two built‑in voices, voice design from text descriptions, and two voice‑cloning modes, achieving a 1.44 % word error rate on the Freya‑TR‑Eval benchmark and competitive intelligibility with far fewer parameters. Practical insights include improved cloning accuracy with reference audio conditioning, optimal 20‑30 s reference length, and rapid low‑rank adapter training for speaker similarity, along with inference optimizations that reduce first‑audio latency to 32 ms.
The paper details a multi‑month engineering effort to build Sophea, a production Greek‑English automatic speech recognition system. It evaluates the system against nine production gates—including word error rate, language identification, and hallucinations—across twenty‑three training iterations and two model architectures, finding that no single training‑data composition satisfies all gates. The authors describe a six‑stage data pipeline that dramatically reduces discarded Greek audio, isolate a hallucination defect to a specific training package, and demonstrate that a three‑model ROVER ensemble achieves full gate coverage and a 29 % relative WER improvement. A separate learned arbiter model is released on the Open ASR Leaderboard with 4.26 % average WER on English tests and 25.88 % on live Greek noisy traffic.
InteractionBench is a benchmark that evaluates real‑time voice systems on four dimensions—conversational timing, spoken‑task accuracy, whispering and volume, and groundedness—using the same recorded user audio across thirteen scenarios. It tests seven systems (GPT‑Live, GPT‑Realtime, Gemini Live, Moshi, PersonaPlex, Freeze‑Omni, and a turn‑based cascade) over 1,519 runs, revealing correlations between timing and task accuracy and highlighting differences in whispering, grammar correction, and groundedness performance. The study also notes protocol variations and the impact of network delay on timing scores.
O'ido is an inference engine that performs open‑vocabulary English speech recognition entirely on an ESP32‑S3 microcontroller, running a 13‑million‑parameter Conformer‑CTC model with int8 quantization and a 1.3‑million‑parameter language model. On LibriSpeech it achieves 3.70%/8.23% WER on test‑clean/test‑other, only 0.12 points below full precision, and outperforms the vendor’s recognizer and several laptop‑based tiny models. The firmware processes audio in about twice the duration of the input on a single core, with a streaming variant that delivers partial transcripts while speaking, though it is not yet real‑time. whyItMatters":"This work demonstrates that sophisticated, open‑vocabulary speech recognition can run on inexpensive, low‑power microcontrollers without dedicated neural accelerators, expanding the reach of voice interfaces to devices that previously could only support keyword spotting or fixed command sets."
arXiv:2609.30483v1 Announce Type: cross Abstract: Audio language models state numbers for acoustic quantities, and neither human opinion nor a judge model says whether such a number is true of the si...