V-GIFT: Boosting Visual Instruction Tuning with Self-Supervised Guidance

The paper introduces V‑GIFT, a lightweight method that enhances visual instruction tuning for multimodal large language models by incorporating a small set of visually grounded self‑supervised tasks expressed as natural language instructions. These tasks—such as rotation prediction, color matching, and cross‑view correspondence—are reformulated into image‑instruction‑response triplets, providing supervision that cannot be solved without visual evidence. The approach requires no human annotations, architectural changes, or extra training stages, and consistently improves vision‑centric performance across various models, regimes, and benchmarks when only 3‑10% of the training data is augmented with these instructions.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computer Vision.

• llms
• fine-tuning
• multimodal
• benchmarks