arXiv Machine Learning August 12, 2026 By Aman Chauhan, Vishnu Pendyala

Benchmarking LLM-Guided Control-Plane Policies for Backend Fault Isolation in HAProxy

Read the original on arXiv Machine Learning →

arXiv:2608. 10532v1 Announce Type: cross Abstract: Static load balancers cannot mitigate a backend that is degraded rather than down: round-robin and least-connections keep routing traffic to a server returning HTTP 500s until an operator intervenes.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

Jul 20

ContinuityBench: A Benchmark and Systems Study of Stateful Failover in Multi-Provider LLM Routing

arXiv:2607. 15899v1 Announce Type: new Abstract: In production large language model (LLM) deployments, high API availability guarantees do not equate to conversational continuity.

By Vishal Pandey, Gopal Singh

llms benchmarks

arXiv AI

Jul 21

Talaria: Session-Aware Serverless Serving of Hundred-Billion-Parameter LLMs

arXiv:2607. 17181v1 Announce Type: cross Abstract: Serverless multi-model LLM systems multiplex popularity-skewed model catalogs over shared GPU pools, yet typically schedule each request independently.

By Utopia Meng, Unicornt Zhao, Derek Li, Goalen Gao, Frank Du

llms agents

arXiv AI

Jun 3

Echelon: Auditable Aggregate-Only Language-Model Adaptation Across Privacy Boundaries

arXiv:2606. 02958v1 Announce Type: cross Abstract: Cross-organization language-model adaptation increasingly faces hard governance constraints: in many deployments, device-level model state-parameters, activations, optimizer state, and per-device updates-cannot be exported outside an administrative boundary.

By Hina Dixit, Punit Kumar, Irene Tenison, Nevasini Sasikumar

fine-tuning safety

arXiv AI

Jul 7

Fine-Grained Computation Offload for Off-the-Shelf Servers in Tens of Lines

arXiv:2607. 02630v1 Announce Type: cross Abstract: Hardware accelerators now sit on the critical path of online serving.

By Bojie Li

efficiency

arXiv Machine Learning

Aug 10

Cascade: Exploiting SLO-Aware latency budget for fair and high goodput LLM inference serving

arXiv:2608. 06557v1 Announce Type: cross Abstract: The reasoning and agentic capabilities of large language models have expanded the range of applications they support, from short interactive exchanges to long, compute-heavy requests.

By Muhammad Adnan, Rohan Mahapatra, Prashant J. Nair, Daniel Berger, Pantea Zardoshti, Rodrigo Fonseca, Esha Choukse

llms agents safety

arXiv AI

1d ago

Task- and Session-Level Model Routing: A Common-Interface Hybrid Evaluation of Four Open-Source Routers Across Four Benchmarks

arXiv:2608. 14641v1 Announce Type: new Abstract: Agentic systems increasingly delegate model selection to a router, yet open-source routers are usually evaluated with different tasks, candidate pools, and execution protocols, limiting direct comparison.

By Kiran N. Kumar, Santhosh K. Saminathan

agents benchmarks

Related stories

ContinuityBench: A Benchmark and Systems Study of Stateful Failover in Multi-Provider LLM Routing

Talaria: Session-Aware Serverless Serving of Hundred-Billion-Parameter LLMs

Echelon: Auditable Aggregate-Only Language-Model Adaptation Across Privacy Boundaries

Fine-Grained Computation Offload for Off-the-Shelf Servers in Tens of Lines

Cascade: Exploiting SLO-Aware latency budget for fair and high goodput LLM inference serving

Task- and Session-Level Model Routing: A Common-Interface Hybrid Evaluation of Four Open-Source Routers Across Four Benchmarks