arXiv Computer Vision
Watch Your Speech: Text-aware Video-to-Speech Synthesis with Textual Conditioning
Read the original on arXiv Computer Vision →The Flow has not summarised this story yet — read it at arXiv Computer Vision.
Related stories
arXiv AI
Sep 16
SyncVoice: Simple and Effective Automatic Video Dubbing with Vision-Augmented TTS
SyncVoice is a new automatic video dubbing framework that adds a lightweight Text‑Visual Fusion Module to a pretrained TTS system, aligning visual features with linguistic representations to produce temporally synchronized speech. The approach avoids complex architectural changes and achieves state‑of‑the‑art performance on the LRS3 dataset in zero‑shot dubbing. When further trained on a large bilingual audio‑visual corpus, SyncVoice improves vocal fidelity while maintaining synchronization, enabling a single model to dub both Chinese and English videos.
Hugging Face Trending Papers
Aug 6
Vorch-Omni: Multi-Task Orchestration of Sight and Sound
Recent advances in generative video modeling have enabled diverse generation, reference-based synthesis, extension, and editing, but existing approaches often rely on fragmented task-specific models. A general model must distinguish heterogeneous target, source, and reference signals to determine what to generate, preserve, or use as guidance, while reducing interference among tasks.
arXiv Computer Vision
Sep 10
Candor-LR: A Dyadic Conversational Dataset for Audio-Visual Speech Recognition
arXiv:2609.10394v1 Announce Type: cross Abstract: Current audio-visual speech recognition (AVSR) benchmarks, like LRS3, rely heavily on clean, scripted and rehearsed speech. They fail to reflect the...
arXiv Computer Vision
Sep 10
AVSRBench: A Multi-Condition AVSR Benchmark
arXiv:2609.10366v1 Announce Type: cross Abstract: While AVSR has achieved sub-1% word error rates on the standard LRS3 benchmark, its reliance on broadcast speech obscures whether this reflects true...
arXiv Computer Vision
3d ago
ControlFoley: Unified and Controllable Video-to-Audio Generation with Cross-Modal Conflict Handling
arXiv:2604.15086v3 Announce Type: replace-cross Abstract: Recent advances in video-to-audio (V2A) generation enable high-quality audio synthesis from visual content, yet achieving robust and fine-gra...
arXiv Machine Learning
Jun 26
Conditional Flow Matching for Visually-Guided Acoustic Highlighting
arXiv:2602. 03762v4 Announce Type: replace-cross Abstract: Visually-guided acoustic highlighting seeks to rebalance audio in alignment with the accompanying video, creating a coherent audio-visual experience.