The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.
The paper introduces an observable evidence standard to assess whether adding a planning capability in industrial AI systems delivers operational value. By separating opportunity, plan change, consequence, equivalence, uncertainty, and scope, the authors evaluate diverse planning additions on real retail datasets, revealing that composite policies can mask negligible contributions, execution changes may not always yield benefits, and indirect evidence alone cannot fully attribute value. The study demonstrates how this evidence record can guide decisions on validation, deployment, monitoring, redesign, and retirement of planning capabilities.
The paper introduces an evidence‑guided recovery framework for AI agents that encounter ambiguous tool outcomes and partial execution, such as timeouts or failures. It defines a shared admissibility check for key expiry, authorization timing, dependencies, and residual effects, and a terminal certificate to distinguish actual goal satisfaction from publicly supported safe completion. In a finite study of 37 assignments, the approach achieved 28 certified completions, and in a larger comparison of 58 paired units, evidence‑guided advice certified 30 completions versus 9 for evidence prompting, demonstrating a significant improvement in task completion rates.
The paper investigates how agent harnesses—specifically planning guidance, execution organization, and completion verification—affect performance in retail and airline pilot tasks. By comparing fixed, task‑specific plans to shuffled policy text of equal length, the study finds that fixed plans improve success rates by about 7 percentage points, especially on complex tasks. A read‑only verifier rejects a majority of invalid episodes while incurring minimal cost, and its impact varies with the penalty for erroneous acceptance, often matching the full planning‑plus‑verification benefit at a lower cost.
The paper investigates how the distribution of capacity and task information across stages of AI production affects the economic value of inference. Using controlled workflow experiments on software‑engineering tasks, it finds that direct execution achieves a 59.6% success rate at token ceilings of 12,000 and 24,000, while information‑constrained planning improves from 36.2% to 51.2%. The study also shows that giving planners access to task issues boosts success, and that scaling token limits changes the balance between planning and execution workloads.
arXiv:2606. 29878v1 Announce Type: new Abstract: Predictive models are increasingly embedded in operational decision-making, yet standard explanation methods typically explain forecasts rather than the decisions those forecasts induce.
Predictive models are increasingly embedded in operational decision-making, yet standard explanation methods typically explain forecasts rather than the decisions those forecasts induce. This distinction is important in predict-then-optimize systems: large forecast changes may leave the optimizer's action unchanged, while small changes can alter the selected decision and its realized value.