arXiv Computation and Language
Sep 23

SpecialEduBench: Benchmarking Vision-Language Models on Knowledge, Skill, and Attitude in Language Intervention for Autistic Children

SpecialEduBench is a new benchmark for vision‑language models that evaluates their pedagogical competence in language intervention for autistic children across knowledge, skill, and attitude dimensions. It contains 4,537 knowledge items, 200 skill items, and 68 attitude items derived from recorded interventions, with 192 response cells for attitude items that combine pressure and monitoring. Eight leading vision‑language models were tested, none achieving full performance, especially in honesty cells, indicating that current models still struggle with situated teaching tasks.

By Jihoi Na, Taeyeong Kim, Sungjune Kong, Jaemin Jung, Min Joung Park, Kyungtae Joo, Ahhyun Kim, Shim Jaechang, Sooyoung Joo, Dongjin Ka, SeJoong Kim, Jimin Kim, HyunJin Jung, Unggi Lee
arXiv AI
Aug 7

Visual Grounding in Zero-Shot Vision-Language Control

arXiv:2608. 06154v1 Announce Type: cross Abstract: Vision-language models (VLMs) are increasingly used as zero-shot controllers, but successful trajectories do not necessarily show that decisions are grounded in visual input: simulator dynamics and conservative action priors can produce favourable scores without meaningful perception.

By J. de Curt\`o, Dayani Plasencia, Diego S\'anchez, I. de Zarz\`a
arXiv Machine Learning
Oct 7

Have I Seen Enough? Frozen Video-Language Models Encode Evidence Readiness

The paper investigates whether frozen video‑language models inherently encode a signal indicating whether sufficient evidence has been observed to answer a question. By training linear probes on seven byte‑identical models, the authors demonstrate that these models contain a readable evidence‑readiness signal with AUROC ranging from 0.733 to 0.905, even when the probe is trained without any footage from the benchmark family. The signal is question‑conditioned, remains robust when the model answers incorrectly, and outperforms traditional uncertainty estimators; it can be leveraged as a Readiness Gating policy that improves answer accuracy by up to 9.75 percentage points without extra computational cost.

By Dan Ben-Ami, Kobi Cohen, Chaim Baskin