arXiv AI
VIDA: A Dataset for Visually Dependent Ambiguity in Multimodal Machine Translation
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
4d ago
VISTA-Bench: Benchmarking Multilingual Image Translation with Image-Specific Rubrics
arXiv:2609.37287v1 Announce Type: cross Abstract: Image translation is a fundamental capability of multimodal models for multilingual applications, requiring visual understanding and meaning preserva...
arXiv AI
Jul 1
Visual Semantic Entropy: Do Vision Language Models Recognize Visual Ambiguity?
arXiv:2606. 31407v1 Announce Type: cross Abstract: Vision-language models can produce confident answers on visually ambiguous inputs, resulting in biased predictions.
arXiv AI
6d ago
Improving Visual Sensitivity of LLMs on Multimodal Machine Translation with Metric-based Loss Weighting
The paper proposes Metric-based Loss Weighting to enhance visual grounding in multimodal machine translation. By increasing loss for tokens that benefit from image context—identified via the Point-wise Cross-mutual Information (PCXMI) metric and its Congruency-based variant—the method improves translation accuracy on the CoMMuTE dataset by over 7 percentage points. Experiments fine-tune three pretrained multimodal LLMs across three language directions, showing superior performance compared to standard fine-tuning while preserving overall translation quality.
arXiv AI
Aug 21
A Model-Internal Protocol for Assessing Multimodal Models as Integrated Systems
arXiv:2608. 11907v2 Announce Type: replace-cross Abstract: As Large Vision-Language Models increasingly aim to integrate visual generation and understanding within a single parameter space, evaluating such structural unification in a cohesive manner remains a critical challenge.
arXiv AI
Jul 3
SEPS: Semantic-enhanced Patch Slimming Framework for fine-grained cross-modal alignment
arXiv:2511. 01390v2 Announce Type: replace-cross Abstract: Fine-grained cross-modal alignment aims to establish precise local correspondences between vision and language, forming a cornerstone for visual question answering and related multimodal applications.
arXiv AI
Jul 24
MELLA: Bridging Linguistic Capability and Cultural Groundedness for Low-Resource Language MLLMs
arXiv:2508. 05502v2 Announce Type: replace-cross Abstract: Multimodal Large Language Models (MLLMs) perform strongly in high-resource languages, yet often produce fluent but culturally "thin" descriptions in low-resource settings.