The paper argues that large language model (LLM) providers, constrained by compute, often degrade service during congestion by routing queries to smaller models, cutting reasoning effort, or truncating context. It shows that this practice misrepresents costs because degraded answers can fail, leading to retries that inflate traffic or churn that erodes lifetime value. By modeling inference allocation with newsvendor, retry, and queueing frameworks, the authors derive a ‘shadow price of intelligence’ that quantifies the marginal value of each query, revealing that throttling under congestion acts as a demand lever rather than a cost lever.
arXiv:2606. 03736v2 Announce Type: replace-cross Abstract: We study resource-constrained dynamic pricing when the seller seeks revenue and valid inference about demand at a price fixed before the selling season.
arXiv:2609.01345v1 Announce Type: new Abstract: Inference cascades cut cost by answering most queries with a cheap model and escalating a hard tail to a frontier model that acts as verifier. A natura...
arXiv:2607. 07207v1 Announce Type: cross Abstract: We analyze how four forces restructure the AI industry over 2026-2030: the DRAM/HBM price surge, frontier-capable open-weight models (GLM-5.
arXiv:2610. 19179v1 Announce Type: cross Abstract: We study packet routing in queuing systems where queues act as selfish, myopic agents that use no-regret learning algorithms to select servers with unknown service rates.
arXiv:2609.23886v1 Announce Type: new Abstract: Software delegates more of its branches to models every year: which queue a ticket enters, whether a command is safe to run, whether a claim clears wit...
ERRAND is a new method for budgeted maintenance of agent memory that treats revalidation of stored knowledge as a priced errand competing for scarce actions. It uses an errand index that is single‑peaked, allowing certainty in either direction to cost nothing, and repairs by writing new versions rather than deleting old ones. In experiments across two drifting tool‑use worlds, ERRAND outperforms non‑oracle policies, achieving up to 10.0 percentage points improvement over eager revalidation while using only 11.0% of steps, and it self‑terminates when no budget is imposed.
Runtime compression of serving state trades quality for capacity with no priced guarantee: systems adapt precision on load signals with no soundness statement, and certified approaches budget request-...
arXiv:2608. 07747v1 Announce Type: new Abstract: We study how to share a single conserved capacity budget across many locations and two service classes when demand is uneven, time-varying, and can exceed supply.
arXiv:2608. 15810v1 Announce Type: new Abstract: Runtime compression of serving state trades quality for capacity with no priced guarantee: systems adapt precision on load signals with no soundness statement, and certified approaches budget request-level risk by a union bound over a pre-declared event count.
arXiv:2610.09519v1 Announce Type: new Abstract: Production machine-learning models are derived artifacts of time-bounded training snapshots: a deployed model is a materialized view over a training cu...
The paper introduces Drift‑Aware Sparse Routing (DRS), a method for routing requests in a multi‑model language service while respecting compute, latency, memory, or cost budgets. DRS estimates reward and resource use from a rolling audit window, routes using pessimistic reward and optimistic cost estimates, updates resource shadow prices online, and applies a hard meter before commitment. The authors provide theoretical regret bounds that separate control from statistics, showing how the method adapts to non‑stationary prompt distributions and model changes.
arXiv:2609.00710v1 Announce Type: cross Abstract: An LLM application often sells or internally allocates several service products: a small or premium model, a short or long token cap, and possibly mu...