Context-Dependent Affordance Reports in Vision-Language Models
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
The Flow has not summarised this story yet — read it at arXiv AI.
arXiv:2603. 04419v2 Announce Type: replace-cross Abstract: We characterize the phenomenon of context-dependent affordance computation in vision-language models (VLMs).
The study investigates how different prompt components affect language model responses in psychometric tests. By crossing five distinct baseline personas with five variants of each prompt element—persona wording, task instruction, item wording, and option symbol—the authors measure response shifts using the 1‑Wasserstein distance. Their analysis of 13 small open‑weight language models on the Big Five Inventory and Short Dark Triad reveals that task instruction and option symbol changes often cause more variation than paraphrasing the persona or item, with prompt artifacts explaining over 50% of the variation for many items.
The paper introduces an interventional protocol to assess how vision‑language models (VLMs) explain the impact of missing modalities on their predictions. By comparing the models’ self‑explanations with actual changes observed after restoring missing inputs, the study finds that VLMs routinely overstate the sufficiency of available evidence and underestimate the effect of adding back missing modalities. Across eight open‑weight VLMs and four tasks, the discrepancy between predicted and realized changes is substantial, revealing systematic mischaracterization of modality dependence.
arXiv:2610.02926v1 Announce Type: new Abstract: Standard factual benchmarks assume clean monolingual prompts and exact-match scoring, two assumptions that break simultaneously in real-world multiling...
arXiv:2607. 20092v1 Announce Type: cross Abstract: Contextual entrainment is the tendency of a model to let auxiliary context in its input pull its output, independently of whether that context is relevant, true, or even meaningful.
The study evaluates whether signals from a frozen vision‑language model, InstructBLIP, can reliably indicate answerability in visual question answering. Using a 10K subset of UNK‑VQA, the authors test predictive uncertainty, cross‑modal alignment, visual grounding, visual dependence, internal representations, and learned probes, finding that all signals yield weak discrimination (AUROC 0.50‑0.54). Even interventions that remove question‑relevant visual evidence or increase probe capacity do not substantially improve answerability detection, underscoring that visual dependence and related metrics are not robust proxies for answerability.