arXiv AI
VibeServe: Can AI Agents Build Bespoke LLM Serving Systems?
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Aug 26
Simthesizer: An Agent-Driven Simulation Framework for LLM Serving Systems
arXiv:2608.24650v1 Announce Type: cross Abstract: System-level simulation is an essential tool for exploring the rapidly expanding design space of LLM serving systems, where real deployments remain c...
arXiv AI
1d ago
ServerBench: How can your coding agents design specialized LLM serving systems?
ServerBench is a benchmark that lets large language model (LLM) coding agents design and test specialized LLM serving systems. The benchmark evaluates agents on a single objective—maximizing latency‑qualified token value per dollar of GPU cost—using stateful replay to preserve session dependencies and independent verification to confirm system correctness. In experiments with 11 coding‑agent models, Claude Sonnet 5.5 achieved 2.97 dollars per dollar on Gemma E4B, outperforming the vLLM reference by 4.87×, and the study highlights key design insights such as coordinating capacity with load selection, interpreting performance feedback through relevant experiments, and verifying execution paths for reliable delivery.
arXiv AI
Oct 15
Agentic Architect: An Agentic AI Framework for Architecture Design Exploration and Optimization
Agentic Architect is a new framework that uses large language models to automatically evolve and optimize CPU microarchitectural designs. Starting from a seed design, the LLM proposes code changes, which are compiled, simulated, and evaluated against performance, area, and power constraints, with results feeding back into subsequent iterations. The framework was tested on prefetching, cache replacement, and branch prediction, achieving notable speedups and area reductions compared to state‑of‑the‑art designs.
Hugging Face Trending Papers
Sep 28
AgentPerfBench: A Benchmarking and Evaluation Suite for Inference Performance of Agentic LLMs
AgentPerfBench is a new benchmarking suite designed to evaluate the inference performance of agentic large language models (LLMs) that handle multi‑turn, tool‑using, and context‑expanding tasks. It builds on real traces from agentic benchmarks such as SWE‑Bench and TerminalBench, and generates synthetic profiles that reflect realistic input/output lengths and turn counts. The suite also provides kernel‑level Nsight Compute traces and a multi‑dimensional roofline model to identify hardware bottlenecks and quantify the gap between traditional chat benchmarks and agentic workloads.
arXiv Machine Learning
Jul 10
Tool-Making and Self-Evolving LLM Agents in Low-Latency Systems
arXiv:2607. 08010v1 Announce Type: cross Abstract: Production LLM agents often waste latency and reliability by regenerating code for the same procedural steps on every request.
arXiv AI
Aug 18
From LLM Inference to Agentic Workloads: Characterization and Implications for Serving Systems
arXiv:2608. 15127v1 Announce Type: cross Abstract: Agentic applications are shifting AI serving from isolated model inference to long-running workloads in which LLMs coordinate tools, environments, and persistent state.