arXiv AI
False Floors: LLM Safety Routing Evaluations Break Under Distribution Shift
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Jun 4
Safety Under Scaffolding: How Evaluation Conditions Shape Measured Safety
arXiv:2603. 10044v2 Announce Type: replace-cross Abstract: A safety score earned on a benchmark need not predict how the same model behaves once it is wrapped in an agentic scaffold the benchmark never tested.
arXiv Computation and Language
Aug 21
Stopping and Routing LLM Judge Panels
arXiv:2608. 19802v1 Announce Type: new Abstract: LLM evaluation pipelines often have many candidate judges: general LLM-as-a-judge prompts, reward models, safety classifiers, confidence variants, and task-specific verifiers.
arXiv AI
Jun 11
Layer-Isolated Evaluation: Gating the Deterministic Scaffold of a Production LLM Agent with a No-LLM, Regression-Locked Test Harness
arXiv:2606. 11686v1 Announce Type: cross Abstract: End-to-end task-success is the dominant way to evaluate LLM agents, but one aggregate number tells you that an agent regressed, not where.
Hugging Face Trending Papers
Aug 20
Stopping and Routing LLM Judge Panels
LLM evaluation pipelines often have many candidate judges: general LLM-as-a-judge prompts, reward models, safety classifiers, confidence variants, and task-specific verifiers. The deployment question...
arXiv AI
Jun 2
When Safe Skills Collide: Measuring Compositional Risk in Agent Skill Ecosystems
arXiv:2606. 00448v1 Announce Type: cross Abstract: LLM agents increasingly rely on community-contributed skills that expand an agent's operational capability set.
arXiv AI
Aug 5
Single Canonical Prompts Underestimate LLM Safety's Surface-Form Sensitivity
arXiv:2608. 02665v1 Announce Type: cross Abstract: A benchmark score is a measurement instrument, yet most benchmarks read each item at a single canonical surface form.