The paper introduces GeoSim, a four‑level framework for analyzing how vision‑language models (VLMs) represent low‑level vision tasks. It evaluates hidden‑layer representations across 24 tasks and two VLM paradigms—autoregressive models and diffusion transformers—using global similarity, local geometry, sparse feature decomposition, and topological verification. The study uncovers the organizing principles of low‑level visual representations and highlights their limitations in cross‑task and cross‑model agreement, offering an interpretability lens for assessing latent transferability and diagnosing model‑specific issues.
By Shao-Jun Xia, Huixin Zhang, Zhen Lei, Anlan Sun, Yuner Zhang, Xiaoyang Chen
FORALL-LEAN-AGENT is a frontend‑agnostic framework that enhances auditable reasoning in formal mathematics and software verification. It integrates isolated workspaces, Lean tools, and fresh review processes—including statement comparison, axiom audits, and independent proof checking—to bind verification evidence and reviewer decisions to the same candidate artifact, ensuring traceable acceptance. Evaluations on VeriSoftBench, PutnamBench, and the Lean Eval track show that the framework improves benchmark success rates and reduces costs while providing evidence beyond aggregate solve counts.
By Naing Oo Lwin
The paper investigates how large language models (LLMs) encode and retrieve facts based on their order of mention in a context. By measuring changes in hidden states when shifting questions from one fact to another, the authors identify an "ordinal vector" that points to a fact’s position in the list rather than its name. Across several models (Qwen, Gemma, Llama), these ordinal addresses are ordered, steerable, low‑rank, and emerge in mid‑layers, suggesting that LLMs locate facts by their mention order, mirroring aspects of human recall.
By Yufa Zhou
The paper investigates how answer candidates in masked diffusion multimodal large language models (MLLMs) can stabilize before their rationales are fully generated. It distinguishes between retrospective stabilization of the logged candidate and token commitment, and evaluates these two temporal markers across three visual question‑answering benchmarks. The study finds that a large portion of the rationale canvas remains unwritten at stabilization, that reducing block length dramatically lowers this fraction, and that direct prompting instructions can significantly alter accuracy depending on the model and dataset. The authors also use matched‑canvas image ablations to separate visual sensitivity from answer stabilization, concluding that coverage rather than conditional accuracy drives most prompting differences.
By Keuntae Kim, Yong Suk Choi
The paper introduces RelationVGGT, a feed‑forward framework that performs 3D spatial relation segmentation without per‑scene optimization or known camera poses. It combines semantic features from a visual foundation model with geometry‑aware representations from a 3D geometry foundation model, and uses a relation transformer to predict subject‑conditioned, cross‑view relations based on a visual subject and a textual query. The authors also present an automated annotation pipeline built on ScanNet++ with VLMs and LLMs to generate scalable training data for this new task.
By Minsu Kim, Jaesung Choe, Jiwoo Lee, Yu-Chiang Frank Wang, Seon Joo Kim
Madeleine is a system that learns to associate conversational memories with relevant cues by simulating human lives offline with a large language model (LLM). It trains a query encoder to predict residual associations on top of frozen similarity, enabling it to retrieve memories without any online LLM calls. In evaluations on the LoCoMo-Plus benchmark, Madeleine achieves state‑of‑the‑art performance when integrated with HyperMem, matches HyperMem’s score when used alone with far fewer LLM calls and context tokens, and significantly improves T‑Mem while preserving standard QA performance.
By Zhiyun Shi
CineMR is a vision‑language model that integrates cardiac image‑analysis tools to perform quantitative assessment of cine cardiac MRI. It uses supervised fine‑tuning and Group Relative Policy Optimization to learn reliable tool invocation, achieving significantly higher accuracy on a multi‑cohort benchmark than existing medical VLMs. The model demonstrates that tool‑augmented reasoning improves ventricular measurement accuracy by up to 23.7% and is essential for robust quantitative CMR interpretation.
By Kunyang Li, Hai Nguyen, Joshua Lowe, Chenguang Zhao, Peace C. Madueme, Mehdi Hedjazi Moghari, Mubarak Shah, Pegah Khosravi, Yuzhang Zhang
The paper introduces a benchmark for evaluating large language models on trustworthy analysis of earnings call transcripts. It proposes a numeric evidence evaluation method that assesses groundedness without expert annotation, and presents an automated pipeline that builds the ECTs-100 dataset from the top 100 S&P 500 constituents. The study also explores the failure mode of conscious incompetence, where models must recognize insufficient evidence and avoid hallucinations, finding that while groundedness is strong, correctness remains a challenge.
By Yingzhu Zhao, Vlad Pandelea, Han Yuan, Bo Hu, Wuqiong Luo, Li Zhang, Zheng Ma
The paper presents a framework for deploying large language model (LLM) agents in smart manufacturing, pairing each factory module with an LLM-based agent and an MCP tool server that exposes the module’s skills via OPC UA. Three agent architectures—orchestrator, peer‑to‑peer, and monolithic—are evaluated on nine production challenges in a simulated six‑module hexagonal factory, with the monolithic and peer‑to‑peer designs achieving a 93 % mean solve rate and the orchestrator uniquely handling a silent conveyor‑belt fault in all runs. The study demonstrates that LLM agents can reduce programming effort, handle unforeseen runtime faults, and exhibit emergent fault‑diagnosis behavior without explicit failure logic, thereby establishing a reproducible foundation for LLM‑programmed smart manufacturing.
By Kay K\"ohle, Darko Anicic, Thomas A. Runkler, Ren\'e Graf
The paper introduces Temporal Activation Injection (TAI), a post‑processing technique that reinforces temporal representations in Video Large Language Models (VideoLLMs) at inference time. By measuring the temporal divergence vector τ_l— the layer‑wise difference caused by reversing video frame order— the authors find that temporal information peaks at intermediate layers and fades toward the output. TAI injects the peak divergence back into later layers, improving temporal reasoning on three VideoLLMs across four benchmarks without affecting non‑temporal performance.
By Youngwoo Shin, Yusung Ro, Minseo Kim, Junmo Kim
Cog-VADU is a training‑free framework that transforms video anomaly detection into a sequential cognitive reasoning task. It uses Chain‑of‑Anomaly Detection Thought Prompting (CoADTP) to create a recurrent reasoning chain across video segments, preserving temporal memory and distinguishing complex anomalies from high‑motion normal activities. A cross‑modal re‑ranking stage aligns textual rationales with visual embeddings to enforce semantic consistency and temporal coherence, yielding competitive zero‑shot performance on multiple VAD benchmarks.
By Mohd Ubaid Wani, Sara Atito, Josef Kittler, Muhammad Awais
VETO (Video Efficient Token Optimization for Vision Language Models) is a plug‑in that reduces the quadratic cost of visual tokens in long‑video inference by applying dual‑axis compression: an intra‑frame compressor merges semantically similar tokens within each frame, and an inter‑frame compressor merges temporally redundant frames. By first compressing spatial dimensions, VETO lowers the cost of subsequent global temporal matching, surpassing single‑axis methods and achieving up to 45% faster inference on models such as LLaVA‑OneVision‑7B while maintaining or improving accuracy. The approach is universally applicable across LLaVA‑OneVision, InternVL‑2.5, and LongVA, preserving or enhancing zero‑shot accuracy even under extreme token budgets.
By Gueter Josmy Faure, Hao Ping Wang, Min-Hung Chen, Winston H. Hsu
The article reviews existing tools for assessing clinical reasoning in large language models, identifying six key dimensions—problem representation, temporal synthesis, differential and management reasoning, counterfactual reasoning, calibrated uncertainty, and reasoning faithfulness. It finds that no single instrument covers all dimensions, with some aspects reasonably addressed while others, especially faithfulness and counterfactual reasoning, remain weakly evaluated. The authors propose combining current tools through binary rubric items, separate completeness and correctness scores, importance weighting, safety caps, temporal consistency checks, and chance‑corrected reliability reporting, and call for further design work on uncertainty, counterfactual reasoning, and faithfulness in longitudinal free‑text contexts.
By Zhangshu Joshua Jiang, Zina Ibrahim, James T. Teo
arXiv:2505.17613v2 Announce Type: replace
Abstract: Automatically evaluating multimodal generation presents a significant challenge, as automated metrics often struggle to align with human evaluation...
By Jihan Yao, Yushi Hu, Wenyuan Wang, Bin Han, Shangbin Feng, Guang Yang, Yujie Yi, Bingbing Wen, Ranjay Krishna, Lucy Lu Wang, Yulia Tsvetkov, Noah A. Smith, Banghua Zhu
arXiv:2602.08889v2 Announce Type: replace
Abstract: Quantitative risk assessment relies on structured expert elicitation to estimate unobservable properties. The Delphi method produces calibrated, au...
By Tobias Lorenz, Mario Fritz
arXiv:2606.22676v2 Announce Type: replace
Abstract: Refusal on a safety benchmark does not reveal how stable that behavior will remain after model updates. Benign downstream fine-tuning can weaken re...
By Dongyub Jude Lee, Jungseob Lee, Seungyoon Lee, Seongtae Hong, Suhyune Son, Sugyeong Eo, Jaehyung Seo, Heuiseok Lim
arXiv:2606.22826v2 Announce Type: replace
Abstract: Evaluating LLMs across many model variants---quantized, fine-tuned, or deployment-specific---requires running large benchmarks repeatedly, a proces...
By Devleena Das, Rajeev Patwari, Vikram Kumar Bukka, Nithin Kumar Guggilla, Elliott Delaye, Ashish Sirasao
arXiv:2608.28421v2 Announce Type: replace
Abstract: Post training a language model to reason means updating its weights. Supervised finetuning and reinforcement learning both place the acquired capab...
By Vishvesh Bhat
arXiv:2609.33149v2 Announce Type: replace
Abstract: A common principle of effective learning is to practice material that is neither already mastered nor too difficult to permit progress. We ask how...
By Hongbo Chen, Guohua Lu, Ting Dang, Hong Jia
arXiv:2505.22767v4 Announce Type: replace-cross
Abstract: Dialogue with a large language model can lead a person to insight: a sudden change in how they understand a problem. This perspective asks ho...
By Eleni Vasilaki