NovGauge is a new benchmark designed to diagnose large language models’ ability to assess scientific paper novelty. It contains 619 paper pairs and 50 multi-paper sets, each labeled along three dimensions—task, problem, and method—by experts from ICLR reviewer overlap claims and survey co-citations. The study evaluates 18 LLMs, revealing high hallucination rates and weak evidence grounding, with the best model achieving only 43‑72% verified F1 across dimensions.
By Guoqiang Zhang, Kexin Tan, Ming Zhang, Li Ju, Wenqing Jing, Zhonghan Yue, Jiayi Chen, Shiqiang Wu, Shaofan Liu, Yue Zhang, Yuankai Ying, Yang Shi, Tao Gui, Qi Zhang, Xuanjing Huang
The paper introduces AgentZip, a memory compression system tailored for high‑fanout AI‑agent sandboxes. It exploits template‑relative and cross‑sandbox redundancy, expands compression to any profitable page, and shifts overhead control to restore‑time prefetching. The approach aligns compression with LLM waiting periods, achieving up to 8.7× memory reduction and reducing slowdown to 1.40× while preserving most savings.
By Mengming Li, Ceyu XU, Qijun Zhang, Jiangnan Yu, Xiangfeng Sun, Haohui Mai, Zhiyao Xie
The paper introduces TASCO, a test‑time adaptation framework that enhances Large Language Model reasoning by incorporating local stability into confidence‑based adaptation while keeping the model frozen. TASCO optimizes a lightweight task‑level prefix using two perturbation strategies—Random Perturbation for distributional stability and Sharpness‑Aware Perturbation for worst‑case sensitivity—to ensure that high confidence aligns with correctness. Experiments show that TASCO improves reasoning accuracy and token efficiency across various LLMs and benchmarks, and behavioral analyses confirm that it maintains stable confidence without over‑concentrating the predictive distribution.
By Bincheng Gu, Min Gao, Zongwei Wang, Yibing Bai, Yulan He, Junliang Yu
The paper introduces the Power Flexibility Index (PFI) to measure how large language model (LLM) training performance changes when GPU power is reduced. Using 131 training runs on H200 and H100 GPUs, the study finds that LLM jobs have significant but variable power elasticity and identifies telemetry signals that can predict PFI during runtime. The authors demonstrate that allocating power based on PFI maximizes overall token throughput, recovering about 1.5k tokens/s per job under a 30% power reduction, which represents 63% of the gap between equal-weight and perfect-information allocations.
By Philip Colangelo, Charles Dawson, Shayan Sengupta, Ayse Coskun, Varun Sivaram
COBRA‑Skills is a new framework that treats skill optimization for large language model agents as a budgeted sequential problem over a dynamically evolving candidate set. It uses contextual‑bandit prioritization to focus evaluations on promising or informative candidates and refines the skill population based on execution feedback. In experiments across six agent benchmarks and three target models, COBRA‑Skills outperforms existing methods, cuts optimization cost by 55–58 % compared to SkillOpt, and requires only 50 unique optimization examples per benchmark.
By Pingchen Lu, Xiangyi Wang, Xiang Li, Jie Mao, Zikun Qu, Junfeng Luo, Yao Shu, Bryan Kian Hsiang Low, Zhongxiang Dai
The paper proposes a neurosymbolic defense architecture for AI-enhanced Security Operations Centers (AI‑SOCs) that protects against indirect prompt injection via log poisoning. It combines deterministic SIEM decoders as a pre‑filter with NeMo Guardrails for semantic validation, and adds a closed‑loop telemetry system for Human‑in‑the‑Loop visibility. Experimental results mapped to the MITRE ATLAS taxonomy show the approach effectively dismantles promptware kill chains and delivers a resilient, observable defense for next‑generation AI‑SOCs.
By Anna Gazani, Spyridon Kounoupidis, Panagiotis Katsaros, Nikolaos Kekatos, Grigorios Tsoumakas, Georgios Koutidis
DeFiFusion is a dual‑modal framework that detects price manipulation attacks in decentralized finance by jointly analyzing transaction events and smart contract semantics. It encodes fine‑grained temporal and economic features from transactions and extracts contract logic using large language models, then fuses these signals with a Dual‑Modal Projection‑Fusion Transformer. The approach achieves state‑of‑the‑art performance, recalling 222 of 225 known attacks with 96.10% precision.
By Rui Cao, Shaojing Fan, Liming Fang, Yuchan Liu, Yingying Jiao, Zhenguang Liu
The paper introduces ACTMED, a diagnostic framework that combines Bayesian Experimental Design with large language models to emulate real‑world clinical reasoning. ACTMED actively selects the most informative test at each step, using LLMs to simulate patient states and update beliefs without needing task‑specific training data. The authors evaluate the system on real datasets, demonstrating improvements in diagnostic accuracy, interpretability, and efficient resource use while keeping clinicians involved in the decision loop.
By Silas Ruhrberg Est\'evez, Nicol\'as Astorga, Mihaela van der Schaar
The paper presents a runtime monitoring approach for distributed large language model (LLM) agent workflows. It introduces Causal Past Logic (CPL), an extension of PT‑DTL, allowing guards in if‑constructs and while loops to access the latest causally visible events and variables from other lifelines. The authors adapt the knowledge‑vector monitor to the ZipperGen framework and prove that the locally computed monitor value matches the denotational semantics of the guard at each event.
By Benedikt Bollig
The paper introduces no‑box vulnerability analysis, a method that detects security flaws without system access or runtime interaction by examining only the functionality metadata of a target. Using this approach, the authors built MCPSEC to audit Model Context Protocol servers for indirect prompt injection vulnerabilities, evaluating it on 20 servers with 177 tools. MCPSEC identified 143 vulnerable tools, achieving 98.9% recall of verified vulnerabilities, outperforming an LLM baseline.
By Zehua Zhang, Jie Hu, Pratham Hegde, Aditya Maheshbhai Gabani, Souradip Nath, Yibo Liu, Siyu Liu, Hongkai Chen, Hulin Wang, Zhuoer Lyu, Chang Zhu, Divij Handa, Yan Shoshitaishvili, Tiffany Bao, Ruoyu Wang, Adam Doupe
ReactHuman is a physics‑grounded benchmark that tests whether multimodal large language models (MLLMs) can make immediate, safety‑critical decisions in simulated humanoid scenarios involving sudden household hazards. The benchmark includes 17 event families, over 1,000 reproducible scenes generated from 240 Hz rigid‑body simulation, and a five‑metric suite evaluating reactions on reasonableness, safety, and physical grounding. Evaluation of seven MLLMs reveals that reactive safety remains unsolved, with models frequently mishandling hazards, relying on appearance over motion, and missing key interception points.
By Yizhan Li, Jianxin You, Mengyang Xiong, Yinhuan Chen, Zicheng Zhao, Dekun Wu, Dongqing Zhang, Bang Liu
The paper introduces Agentic Share-of-Search (ASoS), a multi‑agent AI system designed to aid sellers in competitive decision‑making within large‑language‑model (LLM) mediated e‑commerce. It automates competitive visibility measurement and root‑cause diagnosis by deploying query agents on leading AI platforms and employing a ReAct‑based diagnostic agent to suggest prioritized merchandising actions. A 100‑trial ablation study demonstrates the prototype’s effectiveness, recovering the ablated signal in 39% of trials (95% CI: 30.0%‑48.8%) and 63.9% in high‑correlation cases, outperforming chance by 5.5×.
By Spandan Ghose Chowdhury
The paper introduces a tail‑risk‑aware scheduling strategy for agentic LLM workflows that decouples readiness from immediate release of model turns. By jointly selecting which ready turn to release and controlling the amount of unfinished work kept in the queue, the method uses a mean‑CVaR objective to adapt to evolving tail risk and online turn‑work estimates. Experiments on real software‑engineering task traces show comparable performance to eager release under light load and a significant reduction in the 95th‑percentile workflow flow time, achieving up to a 3.5× speedup under contention.
By Bochao Feng, Jianjiang Li, Haojie Wang, Lin Qiao, Yinghui Li, Yukun Yan, Jidong Zhai
MOSAIC is a training‑free framework that adapts Graph Retrieval‑Augmented Generation (GraphRAG) to each query by converting query‑specific evidence needs into a bounded policy over seed selection, traversal, stopping, and evidence selection. It keeps the corpus graph, indexes, scoring, grounding, and answer generation shared, while an LLM analyzer tailors the exploration strategy per query. On GraphRAG‑Bench, MOSAIC improves answer correctness by over 5 points on Medical and 4 points on Novel, achieves high evidence recall and context relevancy, and reduces path and evidence evaluations compared to fixed policies.
By EunKyeong Lee, Kyeong-Jin Oh, Jinwon Kim, Hye Woo Lee, Minsang Song, Hyeongjun Jang, Junyoung Youn
LogiMed‑RoB is a new benchmark that tests large language models (LLMs) on hierarchical logical consistency in medical risk‑of‑bias assessments, using 860 randomized controlled trials and 14,820 queries based on Cochrane Risk of Bias 2.0 expert logic. The benchmark evaluates models across four dimensions—Atomic Consistency, Domain Consistency, Aggregation Consistency, and Evidential Faithfulness—revealing a catastrophic error‑compounding effect where high atomic accuracy does not translate to end‑to‑end consistency. Experiments on ten state‑of‑the‑art LLMs show that even top models can fail to deduce correct outcomes in a significant portion of cases, highlighting a gap between evidence retrieval and reasoning.
whyItMatters":"The study shows that high outcome accuracy can mask critical reasoning flaws, emphasizing the need for white‑box logical verification before deploying LLMs in clinical settings."
By Jiayu Huang, Zichen Tang, Qianhui Ling, Zemin Kuang, Haihong E
SemVerBench is a benchmark that evaluates how well large language models (LLMs) understand and apply version-constraint resolution semantics, such as determining whether a version satisfies constraints like ^1.2.3 or >=2.0. The study finds that many models struggle with certain corner cases, with GPT‑5.1 performing poorly while Claude and Opus perform much better. The authors suggest that the failures stem from an activation/application gap rather than a lack of knowledge, and recommend that coding agents delegate version resolution to a dedicated resolver tool.
By Qibai Chen, Zeming Liu
The paper introduces a routing-by-reasoning-need controller for diffusion vision‑language models that dynamically selects early commitment, baseline preservation, or reasoning‑supportive decoding based on trajectory signals such as answer closure, commitment evidence, and representation revision pressure. This training‑free approach avoids a single universal generation length, instead tailoring inference‑time control to the reasoning demands of each question. Experiments on answer‑focused, mixed‑reasoning, and chain‑of‑thought benchmarks show that routed control improves robustness compared to fixed long or short decoding and single‑rule interventions, with benefits not solely due to shorter outputs.
By Yixiang Liu, Zhongxing Xu, Zhonghua Wang, Xiaoying Tang
The paper introduces Bayesian backward reasoning as a label‑free anchor for multi‑agent collective decision‑making. By constructing reverse posteriors from explicit likelihoods, the authors obtain differently factorized approximations of the underlying posterior, reducing shared errors among agents. Using Jensen‑Shannon divergence to rank agents, they propose three aggregation strategies—hard selection (MinJS), soft reweighting (FwdJS), and log‑linear fusion (LogLin)—which consistently outperform baseline methods on the DDXPlus benchmark across five LLM backbones, especially when agents disagree.
By Ken Chen, Wei Wang, Sachith Seneviratne, Hansani Weeratunge, Saman Halgamuge
The paper investigates how large language models (LLMs) such as Qwen, Llama, and Gemma use internal knowledge when answering questions. By performing layer‑wise interventions on the hidden state after the question, the authors compare how different request directions (pair‑conditioned vs. global) and answer types (noun, adjective, code) influence the model’s routing of information. The study finds that the influence of request direction varies across models and layers, with some models showing a sustained routing effect while others do not, highlighting distinct patterns of early readability, causal steering, and later content dependence.
By Wenkang Wei, Yuan Fang, Renhe Jiang, Hong Cheng, Xingtong Yu
arXiv:2609. 11916v1 Announce Type: new Abstract: Camera traps often run in the field on edge hardware with limited or no connectivity, making small, locally-deployable vision-language models (VLMs) -- not frontier-scale ones -- the practically relevant class to evaluate for species identification.
By William Zhou, Mayukha Siripuram, Xiao Yan, Ziqi Liu, Yi Ding