arXiv Computer Vision
A Simulator-Grounded Framework For Constructing Verifiable Muscle-Grounded QA From 3D Tongue Meshes
Read the original on arXiv Computer Vision →The Flow has not summarised this story yet — read it at arXiv Computer Vision.
Related stories
arXiv Computer Vision
Aug 27
A Simulator-Grounded Framework For Constructing Verifiable Muscle-Grounded QA From 3D Tongue Meshes (extended version)
The paper presents a simulator‑grounded framework, 3DTongueQA, that generates verifiable muscle‑grounded question‑answer pairs from 3D tongue meshes. By mapping 11‑dimensional muscle activations to fixed‑topology tongue meshes using the ArtiSynth Badin finite‑element model, the authors produce over 891,000 QA records per language, demonstrating portability across English and Korean. Experiments show that the dataset supports both structured prediction and natural‑language QA, achieving high accuracy with various decoder architectures and robust performance on unseen anchors.
arXiv AI
Aug 18
Large Language Models and their Awareness of Mechanics and Spatial Geometry
arXiv:2608. 14615v1 Announce Type: new Abstract: Large Language Models (LLMs) perform well on established code-generation and mathematical-reasoning benchmarks, but their capabilities in mechanics and spatial geometry, here denoted as mechanical engineering awareness, has not been quantified systematically.
arXiv AI
Jun 3
CoMPAS3D: A Dataset and Benchmark for Interactive Motion
arXiv:2507. 19684v2 Announce Type: replace-cross Abstract: Socially interactive humanoid robots must engage with humans through their bodies, adapting in real time to a partner's movement, intent, and abilities.
arXiv Computer Vision
Sep 18
KoUniTalk: A Lightweight Articulation-Centered Korean-English 3D Talking Face Benchmark
KoUniTalk is a lightweight, articulation‑centered benchmark that unifies Korean and English 3D talking‑face datasets onto a single mesh topology. By retargeting VOCASET and Korean speech‑based 3D data to a shared 1,176‑vertex template, it reduces output dimensionality from tens of thousands to 3,528 dimensions, focusing on the mouth and adjacent lower‑face regions. The benchmark includes 22 speakers, 4,978 sequences, and 642,781 frames, enabling controlled speech‑driven facial articulation training and cross‑dataset evaluation in a compact, identity‑neutral space.
arXiv Computer Vision
Sep 4
M3T: Discrete Multi-Modal Motion Tokens for Sign Language Production
M3T introduces a discrete multi‑modal motion token system for sign language production, addressing the need for non‑manual features such as mouthings, eyebrow raises, gaze, and head movements. The approach couples FLAME’s expressive facial space with SMPL‑X body parameters and uses modality‑specific Finite Scalar Quantization VAEs to achieve high face codebook utilization (99.0%). Trained with an autoregressive transformer and a sign‑to‑text translation objective, M3T outperforms existing methods on three standard datasets, notably improving accuracy on NMFs‑CSL from 49.0% to 58.3% without large‑scale pre‑training.
arXiv AI
Aug 19
Where a New Concept Must Enter: Entry Point Gates Cross-Task Usability in Unified Multimodal Models
The paper investigates how new concepts can be integrated into unified multimodal models (UMMs) by separating generation and understanding objectives through a novel visual entity bound to a single task direction. Experiments show that the effectiveness of cross‑task usability depends on where the concept is injected into the shared computation, with a mid‑stack alignment objective achieving high concept acquisition with minimal loss to overall performance. The study highlights that unified weights alone are insufficient; the two directions must share a semantic format at the entry point for efficient concept integration.