Tool use, function calling, orchestration and the protocols that let models act rather than only answer.
arXiv:2609. 18366v1 Announce Type: new Abstract: Reliable agent evaluation is complicated by automatic harness optimization, which repeatedly uses a released benchmark $B_{\mathrm{rel}}$ to guide a Proposer that edits prompts, memory, retrieval, tools, and control code around a fixed target agent.
The paper investigates whether in-context learning (ICL) in large language model agents reflects genuine recursive reasoning or simply statistical extrapolation. By testing LLM agents in a public goods game with manipulated historical feedback, the authors compare decision quality to a rational expectations equilibrium benchmark. They find that disrupting historical patterns eliminates the benefits of longer context, especially in highly interdependent settings, indicating that ICL behavior aligns more with statistical extrapolation than strategic reasoning.
The paper introduces a text-based, multi-agent version of the board game Clue to test multi-step deductive reasoning in large language models (LLMs). Six LLM-based agents (GPT‑4o‑mini and Gemini‑2.5‑Flash) play turn‑based games, and a tool‑augmented approach uses a structured possibility matrix to convert implicit game state into explicit remaining possibilities, thereby offloading memory and deductive constraints from the agents. The study compares this tool‑augmented method against a baseline to assess its impact on reasoning quality and task success in a strategic reasoning environment.
WARD is a runtime‑adaptive Vision Transformer framework designed for edge AI that combines channel‑wise subnetwork partitioning, reliability‑aware continual learning, and dynamic operating‑mode scheduling. It operates two physically isolated subnetworks across four modes—Full‑Precision, Low‑Power, High‑Reliability, and Adaptive—to balance computational cost and fault tolerance while maintaining uninterrupted inference. Implemented on a lightweight FPGA accelerator with minimal area overhead, WARD achieves a network‑level failure rate of 1.79% under high Bit Error Rates and supports rapid mode transitions within a few clock cycles.
The paper introduces cognitive extensions for a dual‑process language agent, adding an Adaptive Memory Module (AMM) for episodic storage and a Self‑Reflection Module (SRM) for execution‑time validation. Implemented as feature‑flagged extensions, these modules allow controlled ablations on the ScienceWorld benchmark. Results show the full system outperforms baselines, with SRM providing the strongest individual improvement, indicating that execution‑time control is the key bottleneck while episodic memory aids once the runtime loop stabilizes.
The paper introduces XConf, an experiential confidence estimator that augments a language model’s current inference with a record of its past graded episodes. By recalling similar past tasks and reflecting on past outcomes, XConf generates confidence scores without accessing logits or updating weights, achieving superior discrimination and calibration across diverse benchmarks. The method demonstrates significant gains in selective prediction, improving success rates on agent tasks by up to 8.7 points.
HINT-Plan is a new method that integrates human intention prediction into robot task planning by using Vision Language Models to infer high‑level human intentions from third‑person images. These intentions are converted into goal states and combined with hierarchical Scene Graphs to formulate joint task‑planning problems in context‑rich environments. In a photorealistic simulation, HINT-Plan achieved a 69.71% success rate, outperforming baselines by up to 35.29% and reducing functional conflicts.
The paper introduces the Free Inference dimension (dFI) as a new combinatorial measure of environmental complexity for value‑mixture agents in finite meta‑reinforcement learning. It shows that dFI is smaller than the VC‑dimension and relates to the Natarajan dimension, providing PAC‑style generalization bounds. The authors also define a complementary PMS identification dimension and demonstrate that a hybrid strategy—averaging until the first collision and then selecting—achieves optimal performance, supported by grid‑world experiments.
The paper evaluates three cost‑efficient large language models—Gemini Flash 3, GPT‑5.4 mini, and Claude Haiku 4.5—on 992 algorithmic programming tasks written as Java Spring Boot service methods. Across eight configurations (model, coding tool, and prompt variants) the authors generated 7,593 methods, classified them into eight outcome categories, and executed 7,936 requests. While structural compliance was high, only 12.9% of returned answers were correct, and 38.4% of methods failed to compute the value they returned, with the most reliable methods being the least accurate.
The paper introduces DualViewEval, a benchmark compression technique for agent evaluation that jointly uses outcome and process signals to learn an exact-size miniset and predict full-benchmark scores. By analyzing large-scale trajectories, the authors identify six complementary process signals linked to final agent performance. Across five agent benchmarks and five baselines, DualViewEval achieves superior compression (24×–40×) and lower error rates, while also revealing capability differences among agents for efficient development.
Affora is a design system aimed at making software interfaces more readable by computer-use agents while still allowing designers visual freedom and maintaining familiar human workflows. The authors conducted three controlled studies on component implementations, visual variation, and interaction-design principles, using the results to create guidance from individual components to full sites, along with reusable implementations and executable checks. Evaluation on independently authored interfaces showed performance gains where Affora addressed existing deficits, with limited effects elsewhere, and a workflow case suggested reduced interaction cost.
The paper investigates how multi‑agent large language models (LLMs) can correct each other’s mistakes, but also how peer pressure can overturn correct answers. It argues that a safeguard— a ‘brake’ that blocks harmful revisions while allowing beneficial ones— is essentially a correctness probe, and that models’ self‑knowledge (measured by AUROC 0.64–0.89) limits the effectiveness of such a brake. The authors find that even white‑box steering cannot break this ceiling, and that adding information before revision, rather than filtering after, is the more promising approach.
The study evaluates six frontier language models on a two‑agent <log(N)>‑Questions game using Wikipedia lead paragraphs. In each game a questioner must identify a target paragraph with exactly <log2 N> yes/no questions, while an answerer only sees the target and the question and replies with a single word. Across 408 games, the models perform similarly, with Claude Opus 5 winning 28 of 68 games and the top five models showing only marginal differences; win rates decline sharply with larger document sets, following a reliability parameter of 0.928 per question. "whyItMatters":"The results reveal how well language models can communicate under information asymmetry, highlighting that even top models struggle to extract a full bit per question and that reasoning token usage does not strongly predict success."
SVMemAgent introduces a streaming video memory (SVMem) that continuously updates a compact representation of observed frames for online keyframe selection without prior knowledge of video length, query, or future frames. The agent decides at each timestep whether to replace an existing memory frame with a new one or discard it, trained via Group Relative Policy Optimization using task-driven rewards from question-answer pairs. Experiments demonstrate that SVMemAgent outperforms existing online baselines and rivals offline methods, and its learned policy tends to favor frames containing textual information, potentially aiding downstream VideoQA tasks.
The paper introduces agentic-eCAL, an extension of the Energy Cost of AI Lifecycle metric to evaluate multi‑agent AI workflows across the edge‑cloud continuum. By combining a two‑rate energy model with OSI‑layer transport analysis, the authors quantify that inter‑agent text transfer accounts for only 0.25% of total workflow energy, highlighting that the main energy cost lies in additional inference and context processing triggered by communication. The study uses extensive GPU benchmarks on NVIDIA A100/H100 with 16 open‑weight models and 8 orchestration topologies to validate the metric and explore placement implications.
SAiFE-gym is a Python module that offers simulation environments for studying trading in Constant Product Markets with Concentrated Liquidity. It decomposes the microstructure of these markets into interactive components, allowing researchers to model various economic settings. The environments are vectorized for scalability in high‑dimensional reinforcement learning workflows, and the paper demonstrates their usefulness by evaluating RL agents under uncertain market parameters.
PentestChain is a ten‑phase automated penetration testing framework that uses a cost‑aware AI cascade, starting with a local 7B‑parameter Ollama model (qwen2.5‑7b) and then free‑tier OpenRouter and Cerebras models, with a rule‑based fallback. It exposes the entire pipeline through a Model Context Protocol (MCP) server that includes eleven tools. The authors evaluate the framework using standard testbeds (AutoPenBench, Cybench subset, PentestGPT 182‑sub‑task benchmark) and report that the local model keeps paid‑API cost at zero while detecting 26 services and enriching 34 CVEs on legacy targets.
The paper introduces HOPE, a second‑order pruning method for Mixture‑of‑Experts language models that accounts for cooperative interactions between experts. Unlike first‑order methods such as REAP, HOPE derives an objective that provably bounds pruning error and is shown to outperform baselines across three large MoE models, multiple calibration sets, and diverse benchmarks, especially at high pruning rates and on agentic tasks. The results demonstrate that preserving expert interactions allows aggressive compression with minimal performance loss on complex workloads.
FairCompressAgent (FCA) is an agentic framework that unifies fairness-aware pruning, incremental quantization, and sparse low‑rank factorization for FPGA deployment. A language‑model planner selects compression configurations based on model profiles and measured outcomes, while an execution layer handles compression, fine‑tuning, evaluation, and constraint‑based selection. Experiments on Fitzpatrick‑17k with VGG‑11 show FCA can reduce inference tensor storage by 59.54% under accuracy constraints, improve validation average precision, and lower equalized opportunity, achieving similar results to one‑shot planning with fewer candidate evaluations.
The paper introduces Adaptive Agents via Latent Topologies (AALT), a method for active imitation learning that selects demonstrations based on their expected impact on start‑to‑goal connectivity rather than generic information gain. AALT builds a latent topology of hub states and learned behaviors, identifies high‑value bridge demonstrations that can solve many tasks simultaneously, and uses these to condition a diffusion policy for planning. In a simulated UR5e robot retrieval task with 72 start‑goal pairs, AALT achieved 100% success after only three demonstrations, outperforming baselines that required many more queries.