LogiScope‑VQA is a new benchmark dataset for evaluating vision‑language models in logistics hazard identification. It contains 2,476 images, 2,918 videos, and 10,274 VQA pairs drawn from real industrial warehouses, covering 18 core objects and 20 risk types across 39 subtasks. Experiments show that even advanced proprietary models lag behind human experts, highlighting a significant gap in perception, understanding, and reasoning for industrial safety.
By Hanjing Zhou, Mingze Yin, Ying Lian, Jun Ma, Chang-Yu Hsieh, Yanbing Zhou
The paper demonstrates that a single-direction white‑box attack, which projects a ‘refusal direction’ from a language model’s weights, remains effective against a 320B‑parameter mixture‑of‑experts (MoE) model (GLM‑5.3‑Flash). The attack requires only a few hundred contrastive prompts and no gradient training, and it reduces refusal behavior by up to 89 percentage points across seven harmful‑content benchmarks while leaving overall capability unchanged. However, the attack’s impact is distributed across multiple components—attention, dense, and routed‑expert writers—so that only a joint intervention removes most of the refusal ability, and the conventional module‑name matching approach fails to capture this effect in MoE architectures.
By Yi Shi, Tanyu Chen, Kai Shen
The paper introduces SALT, a lightweight post‑training technique that injects span‑level supervision into existing cross‑lingual sentence encoders to enhance token representations. Evaluated on five multilingual token‑level benchmarks, SALT achieves the best overall results on four tasks, surpassing alternative fine‑tuning methods and competitive encoders. Additionally, SALT improves sentence‑level performance on cross‑lingual retrieval and classification tasks, demonstrating the effectiveness of span‑level supervision for both token and sentence representations.
By Guillem Ram\'irez
MetroLLM-Bench is a 955‑case benchmark designed to evaluate language models as the policy layer of transit kiosks across six real metro systems, covering routing, fare calculation, disruptions, accessibility, and adversarial input. The benchmark includes 14 deterministic scoring components (Tier 1) and 8 semantic‑quality components (Tier 2), with a 75/25 split for training‑data generation and held‑out evaluation. Twenty‑six models from six vendors were tested, and a 4B Qwen 3.5 student fine‑tuned via PEFT outperformed GPT‑5.6 on Tier 1 and matched GPT‑5.4 on the combined score, while larger models offered no further improvement.
By Remco Hendriks (Continker)
arXiv:2609.10239v1 Announce Type: cross
Abstract: Graph-based retrieval can improve multi-hop question answering, but existing approaches often incur high query-time costs and produce diffuse, oversi...
By Daniel Alejandro Coll Tejeda, Pedro Garc\'ia L\'opez, Daniel Barcelona-Pons
arXiv:2609.10253v1 Announce Type: new
Abstract: Large language models (LLMs) are increasingly deployed in globally used assistants, yet their default choices in culturally grounded everyday situation...
By Bhuvan Arora, Devesh Saraogi, Sravya Varada, Dhruv Kumar
arXiv:2609.10346v1 Announce Type: new
Abstract: Multimodal large language models (MLLMs) process hundreds or thousands of visual tokens per image, incurring prohibitive inference costs. While existin...
By Haiji Liang, Pengfei Zhou, Zhenglin Wan, Wei Wang, Yang You, Wangbo Zhao
arXiv:2609.10410v1 Announce Type: new
Abstract: The growing complexity of content moderation policies presents a critical challenge for their consistent operationalization. While foundation models po...
By Ayan Majumdar, Shounak Paul, Pushpdeep Singh, Ines Abdelaziz, Sayeh Jarollahi, Seungeon Lee, Krishna P. Gummadi, Ingmar Weber, Abhisek Dash
arXiv:2604.10333v2 Announce Type: replace-cross
Abstract: Young children demonstrate early abilities to understand their physical world, estimating depth, motion, object coherence, interactions, and...
By Khai Loong Aw, Klemen Kotar, Wanhee Lee, Seungwoo Kim, Khaled Jedoui, Rahul Venkatesh, Lilian Naing Chen, Michael C. Frank, Daniel L. K. Yamins
arXiv:2608.06411v2 Announce Type: replace-cross
Abstract: Multimodal large language models (MLLMs) achieve strong performance across diverse vision-language tasks, but their efficiency is limited by...
By Yuyao Sun, Tao Deng, Shuang Li, Deqing Wang, Hao Geng, Minjun Yu
arXiv:2609.05736v2 Announce Type: new
Abstract: LLM tool agents can be improved without retraining by modifying the runtime harness around a fixed model: prompts, tool interfaces, middleware, state h...
By Cen Mia Zhao, Haibo Ruan, Wenjie Chen, Pei-fen Tu, Usman Abbasi, Joel Hesch
RevalExo is a new benchmark for locomotion mode recognition that focuses on functional daily activities performed by older adults and clinical cohorts. It includes 27 participants from three groups—healthy older adults, stroke survivors, and older adults with probable sarcopenia—recorded with lower-body IMUs and, for a subset, synchronized egocentric video. The dataset offers 10.1 hours of frame‑level annotations across 11 locomotion modes, and the authors evaluate unimodal, multimodal, cross‑population, and cross‑modal recognition challenges, finding that sensor fusion improves performance but transitions and generalization remain difficult.
By Diwas Lamsal, Juha Carlon, Reinhard Claeys, Maxim Yudayev, Louis Flynn, Tom Verstraten, David Beckw\'ee, Eva Swinnen, Mihai B\^ace, Bart Vanrumste, Benjamin Filtjens
arXiv:2506.19445v5 Announce Type: replace
Abstract: Motion blur remains one of the most common and visually disruptive degradations in real-world smartphone imaging, yet existing deblurring benchmark...
By Syed Mumtahin Mahmud, Mahdi Mohd Hossain Noki, Prothito Shovon Majumder, Abdul Mohaimen Al Radi, Sudipto Das Sukanto, Afia Lubaina, Md. Mosaddek Khan
arXiv:2601.08654v3 Announce Type: replace
Abstract: Rubric-based text evaluation increasingly relies on large language models (LLMs) as scalable judges, yet frozen black-box models can interpret the...
By Yihan Hong, Huaiyuan Yao, Bolin Shen, Wanpeng Xu, Hua Wei, Yushun Dong
arXiv:2605.13841v3 Announce Type: replace-cross
Abstract: Voice agents are increasingly deployed across enterprise applications. However, no existing benchmark jointly addresses realistic conversatio...
By Tara Bogavelli, Gabrielle Gauthier Melan\c{c}on, Katrina Stankiewicz, Oluwanifemi Bamgbose, Fanny Riols, Hoang H. Nguyen, Raghav Mehndiratta, Lindsay Devon Brin, Joseph Marinier, Hari Subramani, Anil Madamala, Sridhar Krishna Nemala, Srinivas Sunkara
arXiv:2605.21384v2 Announce Type: replace-cross
Abstract: As long-horizon coding agents produce more code than any developer can review, oversight collapses onto a single surface: the automated test...
By Bingchen Zhao, Dhruv Srikanth, Yuxiang Wu, Zhengyao Jiang
arXiv:2609.03829v2 Announce Type: replace
Abstract: Few-shot fine-grained image classification (FSFGIC) aims to classify similar images with limited labeled examples. This work highlights the critica...
By Ruiling Liu, Linyue Zhang, Wenyi Zeng, Jiamiao Lu, Weichuang Zhang, Changming Sun, Zejun Zhang, Xiao Zhao
The study investigates structured disagreement among annotators in temporal laughter localization, revealing that disagreements are not random but exhibit systematic patterns—larger at offsets than onsets, more frequent for chuckles than full laughs, and predictable from event attributes. Re-annotating the SMILE-Temporal benchmark with multiple annotators per video shows that evaluating against a single reference annotation can significantly bias system scores and ranking accuracy. The authors propose a disagreement‑calibrated evaluation using conformally calibrated tolerance bands that better reflect the full annotator distribution.
By Eyal Hanania, Daniel Arkushin, Naveh Ayal, Jonathan Benvenisti, Amos Bercovich, Elie Zemmour, Sahar Froim
The paper argues that relying solely on zero‑shot task accuracy is insufficient for evaluating quantized large language models (LLMs) because accuracy ignores changes in the full predictive distribution. It proposes a distribution‑sensitive framework that measures fidelity loss by computing statistical distances—such as Jensen‑Shannon Divergence and Total Variation Distance—between the full‑vocabulary output distributions of a full‑precision BF16 reference and its quantized counterparts. Experiments across five foundation architectures and four reasoning benchmarks show that these divergence metrics increase with stronger quantization, revealing distributional drift that top‑1 accuracy fails to capture, and suggest that mixed‑precision Q4_K schemes can offer lower divergence than uniform Q4_0 at comparable memory usage.
By Shahzeb Qamar, Lorenz Sparrenberg, Christian Bauckhage, Baha Rababah, Carson Leung, Murat Kantarcioglu, Cuneyt Gurcan Akcora, Rafet Sifa
arXiv:2609.08038v2 Announce Type: cross
Abstract: Smart healthcare monitoring systems require precise action recognition to ensure well-being and timely intervention in critical situations such as fa...
By Diwas Lamsal, Pramod Wickramatilake, Jednipat Moonrinta, Mongkol Ekpanyapong, Matthew N. Dailey