← Back to all news
arXiv Machine Learning By Sidi Chang, Peiying Zhu

High-Value Synthetic Supervision for Parameter-Efficient Adaptation of a Compact Japanese Speech Model

Read the original on arXiv Machine Learning →

The paper presents a synthetic pipeline that converts Japanese care handoffs into six-field structured notes, using 182 synthetic audio clips to adapt a 1.47 B‑parameter audio model. Parameter‑efficient adaptation via rank‑16 LoRA achieves 97.7 % of the full‑fine‑tuning performance while training only 12.4 M parameters (≈0.85 % of the backbone). Both full tuning and LoRA outperform the unadapted model on a synthetic test set, demonstrating that a compact model can learn a narrow audio‑to‑structure transformation from limited synthetic data.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Computation and Language
6d ago

Synthetic-to-Real Code-Switched ASR: A Spontaneous Nepali-English Podcast Benchmark

The paper introduces a new benchmark of 13,044 spontaneous Nepali‑English code‑switched podcast utterances, totaling 15.45 h, and evaluates a two‑stage synthetic‑to‑real adaptation framework for ASR. Stage 1 uses supervised fine‑tuning on 0.95 h of synthetic speech, reducing Mixed Error Rate from 23.40 % to 21.52 % and switch‑point error rate from 35.89 % to 28.77 %. Stage 2 applies Group Relative Policy Optimization with script and transcription rewards, yielding modest but statistically significant further improvements, particularly in English script confusion and switch‑point accuracy. The study also shows that an auxiliary script reward term offers no benefit beyond MER‑only optimization in this setting, and that the synthetic SFT provides the main cross‑domain transfer to spontaneous human speech.

By Firoj Paudel
nlpfine-tuningmultimodalbenchmarks
More like this →
arXiv Computation and Language
Oct 22

From Text to Voice: A Reproducible and Verifiable Framework for Evaluating Tool Calling LLM Agents

The paper introduces a reproducible framework that converts existing text-based tool‑calling benchmarks into audio‑based evaluations using text‑to‑speech, speaker variation, and environmental noise, while preserving original annotations. Experiments on seven omni‑modal models show that performance varies strongly by model and task, with notable gaps between text and voice results. The study also includes a stress test, a reference‑free LLM‑as‑judge protocol, and demonstrates that open‑source judges can match proprietary ones, aiding privacy‑preserving evaluation.

By Md Tahmid Rahman Laskar, Xue-Yong Fu, Seyyed Saeed Sarfjoo, Quinten McNamara, Jonas Robertson, Shashi Bhushan TN
llmsagentsmultimodalbenchmarkssafety
More like this →
arXiv Computation and Language
Oct 16

SITAR: A Situational Distortion Metric for Automatic Speech Recognition Evaluation

The paper introduces SITAR, a situational distortion metric for evaluating automatic speech recognition (ASR) systems. SITAR aggregates situational severity and cumulative extent into a unified score using a bounded recursive formulation, and is implemented as a lightweight BERT-family regressor that requires minimal runtime cost. The authors validate SITAR through extrinsic and intrinsic protocols, showing it tracks capability scaling and provides diagnostic utility across diverse datasets and models.

By Mingzhe Lu, Jinhao Liu, Yunpeng Li, Hang Su, Lichun Fan, Jian Luan
llmsmultimodalbenchmarks
More like this →