The Flow has not summarised this story yet — read it at arXiv Machine Learning.
arXiv:2609.23130v1 Announce Type: new Abstract: Large language model (LLM) inference is evolving from an engine-local optimization problem into a distributed control problem involving reusable state,...
Large Language Models (LLMs) are increasingly used to optimize code for performance, but this study shows that such optimizations often increase energy consumption rather than reduce it. Across 11 open‑source models and 1,000 benchmark problems, inference energy ranged from 0.79 to 8.68 MJ, and 42 of 44 configurations produced code that used 4% to 886% more energy than human‑written reference solutions. Mixture‑of‑Experts architectures and token‑limit constraints can reduce inference energy, yet performance‑optimized code remains energy‑heavier, indicating that speed and sustainability are not always aligned goals.
arXiv:2608. 12915v1 Announce Type: cross Abstract: The rapid growth of LLM inference is shifting sustainability concerns from one-time training to continuous serving, where infrastructure decisions shape energy use, carbon emissions, water consumption, and service quality.
arXiv:2509. 04827v3 Announce Type: replace-cross Abstract: The energy cost of Large Language Model (LLM) inference is rapidly becoming a barrier to sustainable and scalable deployment.
arXiv:2608. 08744v1 Announce Type: cross Abstract: The carbon footprint of any deployed Large Language Model (LLM) accumulates during inference, where repeated use of the model substantially exceeds the one-time cost of fine-tuning.
The paper introduces CarbonRouter, an online framework that jointly selects the model, GPU type, and region for each query in multi‑turn LLM web services to minimize carbon emissions. It incorporates a cache‑aware carbon cost model that accounts for the history recomputation cost when a conversation is routed to a different deployment, and a lightweight gate that keeps a session on its cached deployment when a move back is likely. Experiments on two real‑world conversation datasets show that CarbonRouter achieves the lowest carbon footprint among evaluated policies, reducing CO₂ emissions by 16.1 % on average and up to 23.1 % for longer sessions, while also improving predicted answer quality and estimated latency.