arXiv AI
Jul 8
Agent Step Value: Probing the Observer Effect in Black-Box Traces
arXiv:2607. 04419v2 Announce Type: replace Abstract: Final-answer scores hide which agent transitions helped or harmed a trace.
Agent Step Value: Auditing Evaluator-Channel Reversals in Black-Box Agent Traces
arXiv:2607. 04419v3 Announce Type: replace Abstract: When evaluator-derived step rewards are pooled or compared across scoring channels, their sign is treated as transportable.
Related stories
arXiv AI
Jul 7
Agent Step Value: State-Transition Measurement with State-Grounded LLM Evaluators
arXiv:2607. 04419v1 Announce Type: new Abstract: Most agent evaluations collapse a multi-step trace into a final answer, a success flag, or a trajectory-level score.
arXiv Machine Learning
Jun 18
Online Reward-Punishment Learning from Fixed-Channel Perceptual Event Streams without Environment Rewards
arXiv:2606. 18963v1 Announce Type: new Abstract: We study online reward-punishment learning when the environment provides no scalar reward or evaluative label.
arXiv AI
Jul 29
When Do Agent Loops Mistake Stagnation for Progress? Self-Evaluation Bias and Externally Grounded Verification in Long-Running Autonomous LLM Agent Loops
arXiv:2607. 25152v1 Announce Type: new Abstract: Long-running autonomous agents plan, act, and judge their own completion without human intervention.
arXiv AI
Jun 30
SEVA: Self-Evolving Verification Agent with Process Reward for Fact Attribution
arXiv:2606. 29713v1 Announce Type: cross Abstract: Hallucination is the reliability bottleneck for LLM-based agents, and fact attribution verifiers are the last line of defense -- yet today's verifiers emit only opaque binary labels, leaving agents unable to self-correct and operators unable to audit.
arXiv AI
Jun 15
SkillAudit: Ground-Truth-Free Skill Evolution via Paired Trajectory Auditing
arXiv:2606. 14239v1 Announce Type: new Abstract: Agent skills are structured procedural packages that guide frozen LLM agents in specialized workflows.
arXiv Machine Learning
Jul 31
Exposure is not manifestation: measurement target and output resolution jointly determine which behavioural-faithfulness evaluator wins
arXiv:2607. 09306v3 Announce Type: replace-cross Abstract: Behavioural auditing asks whether a language model behaves as it claims, but detection scores are reported without separating two targets: whether a reply was produced under a behaviour-inducing condition (exposure) and whether the behaviour surfaced in it (manifestation).
arXiv AI
Aug 11
READER: Dynamic LLM Provenance from Query-Varying Interactions
arXiv:2606. 10794v3 Announce Type: replace Abstract: Existing black-box LLM provenance methods achieve comparability by querying every candidate model with the same diagnostic prompts.
arXiv Machine Learning
Aug 11
The Replay Gap: Static Evaluation of Model Switching in LLM Agents Scores the Wrong World
arXiv:2608. 08239v1 Announce Type: new Abstract: LLM routers promise efficiency by matching each request to the cheapest adequate model, and are increasingly applied per step inside multi-step agents.
arXiv AI
1d ago
When Is an Agent Evaluation Over? Outcome Finality and Cross-Unit Separation
arXiv:2608. 14940v1 Announce Type: new Abstract: Current agent evaluations score models on the state visible at the end of a stopped run which they count as one trial.
arXiv AI
Jun 16
Who Drifted: the System or the Judge? Anytime-Valid Attribution in LLM Evaluation Pipelines
arXiv:2606. 15474v1 Announce Type: new Abstract: Continuous evaluation of LLM products relies on a strong LLM judge treated as ground truth: a cheap monitor scores every interaction and a team is paged when the score drifts down.
arXiv AI
Jul 31
Do Latent Channels Actually Communicate? A Causal Audit of Latent Multi-Agent LLM
arXiv:2607. 26773v1 Announce Type: new Abstract: Latent communication in large language model (LLM)-based multi-agent systems (MAS) transmits continuous internal representations instead of text, but greater representational capacity does not establish that the receiver uses task-relevant information.