Vision‑language models (VLMs) are increasingly used as judges to select the best image from a set of generated options, yet their decisions are usually validated only by score agreement with human ratings rather than by the images they return. In this study, the authors audited VLM judges on 300 culturally situated prompts, comparing the returned image with unseen human ratings and with random choice, and repeated each decision with reordered candidates. They found that a 4‑B parameter judge barely outperforms random selection, shows significant position bias, and is highly sensitive to candidate ordering, while an 8‑B judge reduces position bias and outperforms a CLIP similarity baseline but still discards many good decisions when filtered.
By Huichan Seo
The paper introduces SciUtopia, a closed‑loop large‑language‑model simulation framework that models the evolving academic research ecosystem, including research direction, collaboration, publication, peer review, funding, and researcher attrition. Running 61 simulation worlds, the system simulates over 40,000 researchers and 1.2 million LLM‑generated peer reviews, revealing that rejection‑driven resubmission increases reviewer burden, cautious exploration balances citation impact with career success, and resource inequality can arise without early‑funding advantage.
By Yiqiao Jin, Yiyang Wang, Lucheng Fu, Bing He, Siheng Xiong, Yijia Xiao, B. Aditya Prakash, Josiah Hester, Srijan Kumar, James Evans, Jindong Wang
MWOP (Modality-aware Width-wise Operation Pruning) is a method that independently prunes visual‑to‑visual, text‑to‑visual, and text‑to‑text attention paths within each layer of multimodal large language models, and separately selects feed‑forward network channels for visual and textual inputs. It uses a first‑order Taylor criterion to guide pruning, re‑evaluates FFN importance after attention pruning, and applies LoRA‑based recovery training. The approach is paired with path‑sparse Triton attention kernels and compact visual‑side FFN execution to achieve practical acceleration, preserving token sequences while reducing computation.
"whyItMatters":"MWOP achieves a 1.6× prefill speedup on LLaVA‑OneVision‑7B while retaining 99.7% performance, and further boosts token‑compression methods to 2.9× and 2.7× speedups, demonstrating its effectiveness across architectures."
By Xudong Wang, Hao Wu, Haozhe Hu, Peiran Yin, Xinghao Chen, Yunpu Ma, Wei Zhang, Xiaoyu Shen
The paper examines safety routers—systems that route user requests to different language models—and finds that their performance degrades significantly when evaluated under distribution shift. In standard benchmarks, routers appear effective because the best single model is chosen from the same evaluation data, but when the data distribution changes, the routing advantage diminishes or disappears. The study quantifies this bias across multiple safety corpora, showing that routers offer little benefit under realistic shift conditions and that recognition‑based defenses can be undermined by attackers who know the model being used.
By Amit Singh Bhatti, Vishal Vaddina
The paper "Chaining Skills to Hijack LLM Agents" introduces APEX, a method that constructs adversarial skill chains to manipulate large language model agents into performing attacker-selected actions. By exploiting the sequential invocation of open-source skills, APEX creates a record of genuine task progress that falsely signals user approval, enabling downstream skills to carry out malicious actions. Experiments on SkillsBench and GPT‑5.4 show high success rates, and a prompting defense reduces but does not eliminate the threat.
By Tian Dong, Zixuan Ma, Haodong Zhao, Huaien Zhang, Shaofeng Li, Hao Chen
The study evaluates how well large language models (LLMs) can annotate missing metadata in PubChem’s ~2 million bioassays, focusing on BioAssay Ontology (BAO) assay format and detection method fields. It finds that 36 % of assays lack an assay format, 89 % lack a BioAssay type, and over 99.9 % lack any BAO‑mapped format or detection technology term, highlighting a critical sparsity in metadata. Seven open‑source and proprietary LLMs achieve recall ≥0.96 for biochemical and cell‑based assay formats, with similar performance for detection technology, though disagreements rise for under‑represented classes and often stem from inconsistencies in silver labels rather than LLM errors. A qualitative test with an industrial curator shows LLM‑generated evidence can prompt revisions of existing labels, indicating that LLMs can flag potentially mislabeled assays.
By Laura van Weesep, Riccardo Tedoldi, Jens Sj\"olund, Hossein Azizpour, Susanne Winiwarter, Ola Engkvist, Jon Paul Janet, Samuel Genheden, Juan Viguera Diez
The paper introduces a data-driven method to generate natural-language explanations for why some questions are harder than others. It estimates question difficulty using Item Response Theory on large language model responses, then prompts an LLM to propose hypotheses that are validated on held-out data. Experiments on mathematical, logical, and commonsense datasets show that these hypotheses are interpretable, predictive, and can be used to adjust question difficulty causally.
By Peng Cui, Qiaoyuan Zheng, Rudolf Debelak, Mrinmaya Sachan
The paper introduces the Separable Law, a framework that predicts how vision‑language model performance varies with language backbone size and visual token count. By fitting this law to 26 InternVL and QwenVL models across a range of backbone sizes (1B–72B) and image resolutions (224–8K pixels), the authors show that some question types scale predictably with model capacity while others do not. The law also provides a closed‑form rule for allocating compute between backbone size and visual tokens, helping to choose near‑optimal model and image sizes under a fixed budget.
By Xinye Zhao, Yunkai Dang, Yunchen Wu, Wenbin Li
Architectural Sampling is a training‑free technique that improves test‑time scaling for frozen vision‑language models by generating diverse candidates through distinct forward computations. It achieves this by reusing selected blocks of decoder layers, varying block location and repetition count, thereby creating computational diversity without updating weights or adding parameters. Experiments on five Qwen checkpoints across twelve multimodal benchmarks show that this method raises pass@9 by an average of 6.58 percentage points over standard temperature sampling, with early‑layer reuse delivering the strongest gains and lower lexical overlap in the generated candidates.
By Akshit Singh, Shyam Marjit, Wei Lin, Leonid Karlinsky, M. Jehanzeb Mirza
Detecting Inconsistencies in Model Specifications with LLM-as-Verifier Reasoning introduces VeriSpec, a method that audits natural‑language model specifications directly using an LLM as a verifier. VeriSpec extracts structured rules, builds a topic‑guided graph to cluster related rules, and applies LLM‑based reasoning to find inconsistencies. In a case study on the OpenAI Model Spec, it identified 405 rules, validated five inconsistencies, and outperformed five baselines with the highest precision and lowest cost per validated inconsistency.
By Zichen Xie, Mrigank Pawagi, Lize Shao, Yang Hu, Wenxi Wang
This paper reviews blockchain-assisted intrusion detection and prevention systems for IoT and IIoT, noting that prior studies ignore the shift toward modern Endpoint Detection and Response (EDR) and Extended Detection and Response (XDR) architectures and lump blockchain’s roles together. It proposes a three‑axis taxonomy—detection‑system class, blockchain functional role, and response‑automation maturity—to classify recent research (2019‑2026). The authors identify gaps such as the lack of per‑endpoint blockchain‑anchored response loops due to latency, deployment, and community mismatches, and highlight cross‑cutting challenges like consensus latency, post‑quantum cryptography, smart‑contract vulnerabilities, and LLM‑based detection risks, concluding with a research agenda for hybrid on‑chain/off‑chain orchestration.
By Yahya Shahsavari, Sara Rouhani, Kaiwen Zhang
The paper presents an analysis of asynchronous reinforcement learning for large language model post‑training, focusing on the impact of stale rollouts. It derives a convergence bound for GRPO‑style algorithms that balances the gradient estimator’s second moment and bias, and introduces a new group‑mass capping GRPO (GMC‑GRPO) method that reduces delay‑dependent bias. Experiments on Qwen3 models and reasoning benchmarks show that GMC‑GRPO outperforms existing baselines in robustness to large rollout delays.
By Qijia He, Ruinan Jin, Jun Luo, Shaofeng Zou, Yingbin Liang
The paper introduces a new method for aligning cross‑lingual representations in decoder‑only large language models by leveraging the outputs of mixture‑of‑experts (MoE) routers instead of traditional hidden‑state alignment losses. This approach pools router outputs over many tokens, providing more reliable sequence‑level comparisons across languages. Experiments on four open‑source MoEs show that the routing loss not only aligns hidden representations but also improves multilingual performance across a diverse evaluation suite.
By Lucas Bandarkar, Clark Peng, Ahmed Haj Ahmed, Aditi Khandelwal, Nanyun Peng
The paper introduces 3D-Prog, a framework that adapts powerful 2D vision‑language models (VLMs) for reliable 3D understanding, manipulation, and generation. It does so by employing Canonical Coordinate Framing (CCF) to anchor inputs and outputs in a shared Euclidean coordinate system, resolving axis ambiguity, metric scale, and reference issues, and Task‑Adaptive Feedback (TAF) to close the reasoning loop with dynamic, task‑specific feedback. Together, CCF and TAF enable 2D VLMs to perform open‑vocabulary 3D tasks without retraining, yielding consistent, interpretable, and high‑quality results across diverse 3D scenarios.
By Arman Raayatsanati, Sombit Dey, Anna-Maria Halacheva, Jan-Nico Zaech, Luc Van Gool, Danda Pani Paudel
The paper introduces a closed‑loop framework that refines structured policies for imitation learning by using large language models (LLMs) to analyze policy rollouts logged as semantically meaningful tabular data. By prompting the LLM to generate diagnostic code, the method identifies and corrects suboptimalities in the policy structure without human instruction. Experiments on car racing and door opening tasks show up to 15% performance improvement over zero‑shot LLM‑generated structures and a 75% reduction in compute needed to reach the same reinforcement learning performance.
By Feiyu Gavin Zhu, Qi Xu, Zhifei Deng, Zhigang Hua, Luke Simon, Jean Oh, Reid Simmons
DuoMind is a distributed hierarchical framework that enables multi-robot coordination by combining vision-language-action models for low‑level execution with vision-language models for high‑level reasoning and inter‑robot communication. Each robot’s orchestrator processes task instructions, local observations, and messages from peers to generate precise action commands and semantic messages for other robots. The authors introduce RoboPoly, a benchmark of long‑horizon manipulation tasks, and show through experiments that DuoMind improves multi‑robot task performance, with ablation studies highlighting the roles of hierarchical orchestration and semantic communication.
By Hanchu Zhou, Dechen Gao, Hang Wang, Brendan Lynch, Boqi Zhao, Qiyao Ma, Raman Goyal, Junshan Zhang
Hierarchical Continuous Diffusion Language Models (HC‑DLM) combine discrete token generation with a continuous latent trajectory in a single denoising process, addressing the independence bottleneck of parallel decoding in discrete diffusion models. The training objective is derived from a variational bound on token likelihood, and the latent state is the sole persistent generative element, with tokens read out and fed back at each step. Experiments on Sudoku, Countdown, and LM1B show HC‑DLM outperforming both discrete and continuous diffusion baselines in puzzle accuracy and generative perplexity.
By Hui Ren, Zihan Li, Chang Liu, Huidong Liu, Alexander Schwing
KaliBench is a fine‑grained benchmark that evaluates large language models’ ability to translate natural‑language queries into executable command‑line instructions for Kali Linux. It contains 8,504 query–command pairs covering 1,642 tools across 23 capability dimensions and 5 security phases, and uses a deterministic, alias‑aware evaluation pipeline with multi‑stage verification (LLM validation, sandboxed execution, and human refinement). The benchmark also supports runtime‑free verifiable rewards, enabling training improvements that bring smaller models close to the performance of a 685B MoE model.
By Pengfei Li, Naufal Suryanto, Sicheng Zhang, Muzammal Naseer
AstroAgentBench is a seven-family benchmark designed to evaluate agentic planning for space mission tasks such as scheduling, observation planning, constellation design, and relay support. The benchmark requires agents to submit planning artifacts that are verified for schema, timing, geometry, resources, and mission value, with results reported as validity and normalized scores compared to task-specific solver references. Across five LLM agent systems and 35 held-out cases, the strongest systems approach or exceed solver-reference scores on several families, while weaker systems often fail to produce high-value valid plans, and even strong systems lose quality on geometric, product-level, or design-heavy tasks. Failure analyses identify task-contract misformulation and weak solution construction as key issues, and successful runs demonstrate that agent calibration against verifier feedback and adaptive search can improve performance; ablation studies show that procedure injection and memory accumulation help when they provide the missing formulation, calibration, or search support.
By Weiyi Wang, Xinchi Chen, Jingjing Gong, Xuanjing Huang, Xipeng Qiu
PhGPO: Pheromone-Guided Policy Optimization for Long-Horizon Tool Planning proposes a new method for improving multi-step tool planning in large language model agents. The approach draws inspiration from ant colony optimization, learning a trajectory-based transition pattern—referred to as pheromone—from historically successful tool-use paths. This learned pheromone is then used to guide policy optimization, steering the agent toward reusable tool transitions and enhancing long-horizon planning performance.
By Yu Li, Guangfeng Cai, Shengtian Yang, Han Luo, Shuo Han, Xu He, Dong Li, Lei Feng