arXiv AI
Faces in Things, Faces in Nothing: Probing Face Selectivity in Vision Models with Pareidolia
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Jul 22
When Visual Evidence is Ambiguous: Pareidolia as a Diagnostic Probe for Vision Models
arXiv:2603. 03989v2 Announce Type: replace-cross Abstract: When visual evidence is ambiguous, vision models must decide how to interpret face-like patterns.
arXiv AI
Sep 30
Similar Choices, Different Attention: Cross-Modal Associations in Humans and Vision-Language Models
The study compares human and vision‑language model (VLM) responses to cross‑modal association tasks, using identical stimuli (a pseudo‑word and two images) and recording both choices and eye movements. While larger VLMs show some alignment with human choices, their attention patterns correlate poorly with human gaze, performing no better than a simple center‑bias baseline. Fine‑tuning VLMs on human choices improves choice alignment but not attention alignment, and training on human gaze improves attention correlation without affecting choice accuracy.
arXiv Computer Vision
Oct 27
When Pointing Becomes Profiling: Social Bias and Refusal in Text-Steerable Visual Encoders
arXiv:2610.35017v1 Announce Type: new Abstract: Text-steerable visual encoders condition a frozen Vision Transformer on a natural-language prompt and point at the matching image region, producing pro...
arXiv AI
Oct 21
CounterCount: Diagnosing Knowledge Bias in How Vision-Language Models Count Attributes
CounterCount is a diagnostic dataset that pairs factual and counterfactual images to test whether Vision‑Language Models (VLMs) rely on visual evidence or on world knowledge when counting attributes. Humans achieve near‑perfect accuracy on both types of images, whereas VLMs drop from 91.6% to 49.5% on counterfactuals, often defaulting to canonical counts. The study shows that VLMs exhibit a System‑1‑like bias, underutilize visual evidence, and that targeted input manipulations can mitigate this bias.
Hugging Face Trending Papers
Oct 26
When Pointing Becomes Profiling: Social Bias and Refusal in Text-Steerable Visual Encoders
Text-steerable visual encoders condition a frozen Vision Transformer on a natural-language prompt and point at the matching image region, producing prompt-conditioned heatmaps and features. We audit t...
arXiv Computer Vision
Aug 25
EXPL-FR: Explaining Face Recognition Models via Vision-Language Alignment
EXPL-FR is a lightweight adapter that aligns a vision‑language model’s image encoder with a frozen face‑recognition (FR) embedding space, enabling the FR model to be explained using semantic attribute prompts without any text training. By mapping 978 attribute prompts across 22 categories into the FR space, the method identifies the most detectable concepts—forming a readable semantic signature that better separates identities than the full vocabulary. The approach is evaluated on four FR backbones and two VLM encoders, providing identity‑level, per‑image, and differential explanations, and demonstrates that prompt‑driven audits can rank FR models by per‑ethnicity error and attribute‑change verification cost without requiring labeled data.