arXiv AI September 25, 2026 By Nhat-Nam Nguyen, Pierre-Andre Vuissoz, Yves Laprie

Anatomy-aware cross-speaker adaptation of complete vocal-tract acoustic-to-articulatory inversion

The paper introduces a geometric adaptation framework for cross‑speaker acoustic‑to‑articulatory inversion that leverages anatomical landmarks on vertebrae and dental structures. By applying an affine transformation followed by a thin‑plate spline deformation, the method maps predicted vocal‑tract contours from a fixed model to unseen speakers without retraining. Experiments on a single‑speaker rt‑MRI database and eight additional speakers show that the combined affine‑plus‑TPS approach with 12 and 14 landmarks yields the lowest mean point‑to‑closest‑point error of 3.19 mm.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

safety

Hugging Face Trending Papers

Aug 10

Structured Phonological Representations for Audio-Articulatory rtMRI Speech Classification

Real-time MRI makes it possible to observe vocal-tract articulation during speech, but mapping these articulatory patterns to phonetic and phonological categories remains challenging. We investigate whether PhonoQ, an audio-based model trained to recognize structured phonological features, provides useful information for audio--articulatory modeling.

arXiv AI

Jun 16

ArtBoost: Synthetic Articulatory Data Augmentation for Acoustic-to-Articulatory Inversion

arXiv:2606. 16327v1 Announce Type: cross Abstract: Recent acoustic-to-articulatory inversion (AAI) models rely on electromagnetic articulography (EMA) data, which are costly and limited in scale.

By Hyung Kyu Kim, Byungchan Hwang, Hak Gu Kim

fine-tuning

arXiv Computer Vision

Sep 10

Arti-JEPA: Adapting Video World Model to Real-Time MRI of the Vocal Tract for Speech-Production Analysis

arXiv:2609.09757v1 Announce Type: cross Abstract: Real-time MRI (rtMRI) captures the dynamics of the entire vocal tract during speech, but labeled data are scarce and the modality - single-slice, gra...

By Hong Nguyen, Sean Foley, Christina Hagedorn, Yijing Lu, Sudarsana Reddy Kadiri, Dani Byrd, Shrikanth Narayanan

rag fine-tuning

arXiv AI

Sep 12

Flexible and Interpretable Accent Distance Measurements

The paper introduces a method for measuring accent differences that balances interpretability and practicality. It proposes using articulatory representations obtained via articulatory inversion as an interpretable basis for accent comparison, while employing optimal transport to compare accents across any type of recording. This approach aims to overcome the limitations of traditional phonetic analyses and embedding‑based methods, which are either time‑consuming or non‑interpretable.

By Charles McGhee, Mark J. F. Gales, Kate M. Knill

rag multimodal

arXiv Computation and Language

4d ago

Reconstructing the Vocal Tract with Differentiable Acoustic Simulation

arXiv:2609.36737v1 Announce Type: cross Abstract: The vocal tract is the region of the human body responsible for filtering one's voice to create speech. In this paper, we present a differentiable an...

By Eric Ming Chen, Jin Woo Lee, Vincent Sitzmann

arXiv Computer Vision

Sep 18

KoUniTalk: A Lightweight Articulation-Centered Korean-English 3D Talking Face Benchmark

KoUniTalk is a lightweight, articulation‑centered benchmark that unifies Korean and English 3D talking‑face datasets onto a single mesh topology. By retargeting VOCASET and Korean speech‑based 3D data to a shared 1,176‑vertex template, it reduces output dimensionality from tens of thousands to 3,528 dimensions, focusing on the mouth and adjacent lower‑face regions. The benchmark includes 22 speakers, 4,978 sequences, and 642,781 frames, enabling controlled speech‑driven facial articulation training and cross‑dataset evaluation in a compact, identity‑neutral space.

By Hyunjung Chung, Unsang Park

benchmarks