The paper investigates how vision‑language models (VLMs) can be used for training‑free video anomaly detection (VAD) by answering questions about video segments. It shows that the way a VLM’s output is converted into an anomaly score—either by taking only the most likely answer (generated readout) or by using the full probability distribution (probability readout)—has a significant impact on ranking performance. Across four large VLMs, the probability readout consistently outperforms the generated readout, achieving 5–13 point gains in AUROC or AP, because the generated readout compresses the rank order into only a few distinct scores.
arXiv:2608.21244v2 Announce Type: replace Abstract: Anomaly detection aims to identify observations that deviate from normal patterns. Recent work uses pretrained vision-language models (VLMs) for tr...
Probe‑VAD introduces an ordinal binary‑probing framework that leverages frozen vision‑language models for training‑free video anomaly detection. By querying ten ordered severity thresholds and extracting YES/NO continuation likelihoods, it builds a cumulative severity profile that is converted into a continuous anomaly score with isotonic projection for ordinal consistency. Experiments on public benchmarks show that this simple interface yields superior performance at low computational cost, avoiding the limitations of caption‑based compression or restricted numerical scoring.
The paper introduces an audit layer for vision‑language model benchmarks that reuses per‑question artifacts from evaluation runs to assess the benchmark itself. Six detectors analyze saved outputs, and an additional text‑only rerun removes visual inputs to produce per‑question reports and aggregate statistics, including permutation‑null excess and interval estimates. Applying this method to the MMStar benchmark reveals that 20.2% of its questions show no accuracy gain from images, with a small but significant excess in mathematics and science categories, and identifies questions where all models chose the same non‑gold option.
The paper proposes a lightweight federated multiple‑instance learning (MIL) framework that trains only a compact MIL scorer across distributed clients while using a frozen vision‑language model (VLM) to verify high‑scoring video segments post‑hoc. Two VLM feedback interfaces are explored: a parsed text‑generation interface and a logit‑based interface that derives a continuous anomaly score from next‑token Yes/No probabilities. Experiments on UCF‑Crime with InternVL3.5‑2B and Qwen3‑VL‑2B‑Instruct show that the logit interface consistently improves frame‑level AUC and AP over the MIL baseline without requiring temporal post‑processing, whereas the text‑generation interface is more sensitive to prompts, parsers, and model choice.
The paper shows that chain‑of‑thought (CoT) instructions can distort evaluation of vision‑language models (VLMs) when a scorer reads answer‑label logits before the model generates a rationale. On ScienceQA, Qwen2.5‑VL‑7B’s accuracy falls from 80.76% to 45.48% under this CoT‑prefix scoring, and most predictions incorrectly pick the first option. Linear probes and free generation recover most of the lost accuracy, indicating that the answer information remains in the hidden states but is missed by the early readout. The authors explain the mismatch with vocabulary and layer diagnostics, noting that probability mass shifts toward continuation tokens while answer information stays linearly accessible in later layers. The effect varies across datasets and models, but the study demonstrates that CoT‑prefix scoring can misrepresent model knowledge unless the requested and scored outputs are aligned.
arXiv:2609.13288v1 Announce Type: new Abstract: Video-language models can answer multiple-choice questions with high confidence yet be wrong. We study whether answer-level reliability scores can be i...
arXiv:2610.28560v1 Announce Type: new Abstract: Visual anomaly detection aims to detect abnormal objects, while anomaly samples are often scarce during model training. Assembling multiple pretrained...
arXiv:2607. 13305v1 Announce Type: cross Abstract: Benchmark accuracy in video large language models (LLMs) is often treated as evidence of visual understanding.
arXiv:2610.19823v1 Announce Type: new Abstract: Text-to-video (T2V) generation models can produce visually realistic videos that nevertheless violate the physical consequences implied by the prompt a...
arXiv:2610.29241v1 Announce Type: new Abstract: In this paper, we investigate why vision-language models (VLMs) produce errors by decomposing them into two sources: visual-representation failures, in...
The paper investigates how the scoring rules used in multimodal large‑language‑model (LLM) benchmarks can dramatically alter reported accuracy. By reproducing official scorers for eight benchmarks and comparing them to a blinded human‑plus‑LLM adjudication, the authors show that the same model outputs can yield accuracy differences of up to 22 points depending on the rule. They argue that no single scoring rule is reliable and recommend that benchmarks report non‑compliance rates and use at least two distinct scoring rules.
The paper introduces VideoRouteAudit, a controlled experiment that isolates the effects of frame selection, evidence injection, and expert routing in long‑video question answering. On a large Video‑MME benchmark, no frame selector outperforms uniform sampling, and the gains from evidence injection or model swapping are largely attributable to the specific model or data ordering rather than the selection strategy itself. The study also corrects earlier results, showing that ASR transcripts can either help or hurt performance depending on placement, and that swapping the answering model yields the largest performance contrast.