arXiv AI By Xinyu Mao, Junsi Li, Chenyang Liu, Haoji Zhang, Ming Sun

Whose record is this? Diagnosing and authorizing record use in personalized multimodal models

Read the original on arXiv AI →

The paper introduces RecordAuth-Diag, a diagnostic suite of 3,690 cases that tests whether a visual personalization system correctly authorizes the use of a record by checking subject presence, record-edge validity, and answer support. Violations, termed visual memory misbinding (VMM), are identified by manipulating the image–record edge while keeping other inputs constant. The study evaluates several multimodal models (Gemma, Qwen, Phi, CoViP) and shows that typed pre‑generation authorization can dramatically reduce unauthorized record exposure while affecting recall.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI
Sep 28

The Uncontrolled Variable: Vision-Language Refusal Is Conditioned on the Image-Attachment Interface, and Not Robust to Irrelevant Image Properties

The paper demonstrates that vision‑language models’ refusal behavior is heavily influenced by whether an image is attached to a request, even when the image is blank or unreadable. Attaching such an image shifts refusal scores by large margins for borderline‑benign prompts while leaving genuinely neutral instructions largely unchanged. This effect varies with image properties, persists across checkpoints, and is not mitigated by explicit instructions to ignore the image.

By Haoyu Zhang, Yi Feng, Hanwen Liu, Shibo Zheng, Zhuoxi Wang, Yang Chen, Haowen Xu, Xiangchen Guan, Mohammad Zandsalimy, Shanu Sushmita