Lightweight, Rubric-Guided Trajectory Evaluation for Production AI Agents

LiteTrajEval is a lightweight architecture designed for budget‑bounded trajectory evaluation of large‑language‑model agents. It creates compact domain‑specific rule profiles offline, preprocesses trajectories online to flag heuristic failure signals, and then uses a single rubric‑guided LLM judge to generate structured diagnostic reports. On public Magnetic‑One‑style and τ‑bench‑style datasets, LiteTrajEval improves failure‑localization alignment with human annotations by 20–35 percentage points and reduces cost and evaluation time by 6× and 8× respectively, and has been deployed in an enterprise agentic platform.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

• llms
• agents
• safety