arXiv Computer Vision
When Pointing Becomes Profiling: Social Bias and Refusal in Text-Steerable Visual Encoders
Read the original on arXiv Computer Vision →The Flow has not summarised this story yet — read it at arXiv Computer Vision.
Related stories
Hugging Face Trending Papers
Oct 26
When Pointing Becomes Profiling: Social Bias and Refusal in Text-Steerable Visual Encoders
Text-steerable visual encoders condition a frozen Vision Transformer on a natural-language prompt and point at the matching image region, producing prompt-conditioned heatmaps and features. We audit t...
arXiv Machine Learning
Jun 15
Gaze Heads: How VLMs Look at What They Describe
arXiv:2606. 14703v1 Announce Type: cross Abstract: How a vision-language model internally solves the task of describing an image is far from obvious.
arXiv Computation and Language
Sep 25
Free the Language Model From the Vision Encoder: Semantic Serialization as a Perception Interface for Small Language Models
The paper introduces a perception interface that separates vision from language in vision‑language models. A frozen perception stack detects objects, a deterministic semantic serializer converts the perceived state into text, and a standard text‑only large language model (LLM) answers questions. Experiments on a campus‑robot benchmark show that this serialized interface outperforms a zero‑shot VLM of the same language‑model size, especially as the language model shrinks, and that the advantage persists under paraphrase and different supervision regimes.
arXiv Machine Learning
Sep 3
FairLens: Benchmarking Fairness in Vision-Language Models for High-Stakes Decision-Making
FairLens is a benchmark and evaluation framework that measures fairness and validity of vision‑language models (VLMs) in high‑stakes domains such as hiring, legal, and healthcare. It uses over 100,000 face‑image and question pairs covering gender, race, and age, and assesses responses through demographic parity, soundness, demographic association, and bias in free‑text generation. The study finds that VLMs often make unwarranted inferences from faces rather than abstaining, especially in legal and healthcare contexts, and that small parity gaps can still hide unsafe treatment across groups.
arXiv Computer Vision
Oct 12
What Do Visual-Head Discovery Scores Actually Find? Attention Source, Necessity, and Random-Control Pitfalls in Vision-Language Models
The paper investigates how different design choices in visual-head discovery scores affect the attention heads selected in vision‑language models. By systematically varying query type, discovery corpus, and attention source in Qwen3‑VL‑8B‑Instruct, the authors find that the attention source and query type have a larger impact on head selection than the corpus. While necessity scores distinguish discovery settings, steerability does not show a significant advantage over random head sets.
arXiv AI
Oct 1
Caption-Mediated Perceived-Safety Estimation for Pedestrian Routing
The paper introduces an explainable pedestrian routing method that estimates perceived safety by first generating a natural‑language caption from street‑level images and then deriving risk scores solely from structured features of that caption. Benchmarking nine captioning setups against a CLIP image‑embedding baseline shows comparable performance, and the system was deployed on over 650,000 images across 36 wards in Manchester and Huddersfield. Field validation with 3,669 ratings from 70 participants revealed a modest but statistically significant correlation (r = 0.262) with human judgments, while a stronger supervised benchmark did not translate into better real‑world performance.