The Shadow Price of Intelligence: Quality Degradation in LLM Inference as a Supply Chain Problem

The paper argues that large language model (LLM) providers, constrained by compute, often degrade service during congestion by routing queries to smaller models, cutting reasoning effort, or truncating context. It shows that this practice misrepresents costs because degraded answers can fail, leading to retries that inflate traffic or churn that erodes lifetime value. By modeling inference allocation with newsvendor, retry, and queueing frameworks, the authors derive a ‘shadow price of intelligence’ that quantifies the marginal value of each query, revealing that throttling under congestion acts as a demand lever rather than a cost lever.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

• llms