arXiv Machine Learning July 2, 2026 By Zewen Liu

EPC: A Standardized Protocol for Measuring Evaluator Preference Dynamics in LLM Agent Systems

Read the original on arXiv Machine Learning →

arXiv:2607. 00297v1 Announce Type: new Abstract: When LLM agents use evaluator feedback to adapt their behavior in closed loops, evaluator biases propagate through the agent's strategy distribution -- a phenomenon known as evaluator preference coupling.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv AI

Jul 1

Calibrating the Evaluator: Does Probability Calibration Mitigate Preference Coupling in LLM Agent Feedback Loops?

arXiv:2606. 31371v1 Announce Type: cross Abstract: When large language model (LLM) agents adapt their behavior through evaluator feedback, systematic evaluator biases propagate into the agent's learned strategy distribution - a phenomenon termed evaluator preference coupling.

By Zewen Liu

llms agents

arXiv Computation and Language

Sep 23

Measuring the Serving Stack Instead of the Model: Hidden Confounds in Local Tool-Use Evaluation

arXiv:2609.26693v1 Announce Type: new Abstract: A coding agent must emit a valid tool call--a parseable invocation of a tool in the provided schema--before the harness can execute its chosen action....

By Lijuan Tang, Yuemeng Zheng

llms agents

arXiv AI

Jun 17

SEAGym: An Evaluation Environment for Self-Evolving LLM Agents

arXiv:2606. 17546v1 Announce Type: new Abstract: Self-evolving LLM-based agents improve mainly by changing their agent harness: the structured execution layer around a base model, including prompts, memory, tools, middleware, runtime state, and the model-tool interaction loop.

By Congjie Zheng, Chuanyi Xue, Bin Liang, Jun Yang, Changshui Zhang

llms agents benchmarks

arXiv Machine Learning

Jun 30

A Diagnostic Framework and Multi-Evaluator Audit of Evaluator-Driven Preference Dynamics in Self-Adapting LLM Agents

arXiv:2606. 29719v1 Announce Type: new Abstract: Measurements of proprietary LLM evaluators can become invalid within weeks -- we document one case and provide the diagnostic framework to detect it.

By Liu Zewen

llms agents multimodal benchmarks

arXiv AI

Jul 3

A Unified Framework for the Evaluation of LLM Agentic Capabilities

arXiv:2605. 27898v2 Announce Type: replace Abstract: As LLMs are increasingly deployed as agents, reliable assessment of their agentic capabilities has become essential.

By Pengyu Zhu, Lijun Li, Yaxing Lyu, Qianxin Luo, Jingyi Yang, Yi Liu, Tingfeng Hui, Xinyu Yuan, Li Sun, Sen Su, Jing Shao

llms agents benchmarks safety

arXiv AI

Sep 2

UniACE: A Unified Framework for Evaluating LLM Agentic Capabilities

UniACE is a unified framework that standardizes the evaluation of large language model (LLM) agents by representing each benchmark as an instruction–tool–environment triplet and running models through a shared, task‑agnostic harness in isolated runtimes. It preserves native success criteria, offers an offline mode for dynamic‑resource tasks, and standardizes efficiency metrics, execution records, and failure attribution. Applying UniACE to 7 benchmarks across 24 domains and 15 models revealed significant score shifts, ranking reversals, and sensitivity to evidence representation, highlighting the impact of evaluation configuration on reported agent performance.

By Pengyu Zhu, Lijun Li, Yaxing Lyu, Qianxin Luo, Jingyi Yang, Yi Liu, Tingfeng Hui, Xinyu Yuan, Li Sun, Sen Su, Jing Shao

llms agents benchmarks

One email a morning, machine-written

Related stories

Calibrating the Evaluator: Does Probability Calibration Mitigate Preference Coupling in LLM Agent Feedback Loops?

Measuring the Serving Stack Instead of the Model: Hidden Confounds in Local Tool-Use Evaluation

SEAGym: An Evaluation Environment for Self-Evolving LLM Agents

A Diagnostic Framework and Multi-Evaluator Audit of Evaluator-Driven Preference Dynamics in Self-Adapting LLM Agents

A Unified Framework for the Evaluation of LLM Agentic Capabilities

UniACE: A Unified Framework for Evaluating LLM Agentic Capabilities