arXiv AI
When does a scaling result justify a different allocation? A critical review of resource-allocation evidence for AI systems
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Aug 7
HarnessOpt-Bench: Evaluating LLMs at Harness Optimization
arXiv:2608. 06301v1 Announce Type: new Abstract: As LLMs are increasingly deployed within agentic systems, their capabilities depend not only on the model weights but also on the harness: the prompts, tools, control flow, memory, and orchestration code surrounding them.
arXiv AI
Sep 18
The Organization of Inference: Information, Resource Constraints, and AI Production
The paper investigates how the distribution of capacity and task information across stages of AI production affects the economic value of inference. Using controlled workflow experiments on software‑engineering tasks, it finds that direct execution achieves a 59.6% success rate at token ceilings of 12,000 and 24,000, while information‑constrained planning improves from 36.2% to 51.2%. The study also shows that giving planners access to task issues boosts success, and that scaling token limits changes the balance between planning and execution workloads.
arXiv AI
Jun 17
How Inference Compute Shapes Frontier LLM Evaluation
arXiv:2606. 17930v1 Announce Type: new Abstract: AI evaluations are shifting toward harder tasks that benefit from longer trajectories involving tool use and iterative problem solving.
arXiv AI
Jul 15
Do AI Agents Know When a Task Is Simple? Toward Complexity-Aware Reasoning and Execution
arXiv:2607. 13034v1 Announce Type: new Abstract: Large language model (LLM) agents increasingly automate multi-step engineering and informatics workflows, yet they rarely ask how much effort a task actually requires.
arXiv AI
Jun 4
Not All Errors Are Equal: Consequence-Aware Reasoning Compute Allocation
arXiv:2606. 04402v1 Announce Type: new Abstract: Modern reasoning models can allocate different amounts of test-time computation, such as thinking tokens, model calls, or compute budget, to different tasks.
arXiv AI
Aug 11
The Scaling Paradox in Human-AI Collaboration
arXiv:2608. 00818v2 Announce Type: replace Abstract: The discovery of scaling laws has highlighted the extraordinary potential of AI systems with a striking empirical pattern: as AI systems scale, their capabilities tend to improve predictably.