Mitigating Saliency Collapse: Robust Saliency-Aware Long-Text Image-Text Alignment

The paper introduces RoSA, a saliency-aware fine‑tuning framework for long‑text image‑text alignment that addresses the failure mode of saliency collapse. RoSA decomposes image‑text pairs into salient and contextual views using a modality‑independent strategy that combines visual saliency estimation with LLM‑based text decomposition, and employs auxiliary objectives to align these views. The method improves long‑text retrieval performance and robustness to saliency collapse, and its auxiliary objectives can be added to existing tuning methods for consistent gains.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

• llms
• fine-tuning
• multimodal
• benchmarks
• safety