arXiv AI
NetConfArena: An Executable Benchmark for LLM Agents in Closed-Loop Network Configuration
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Jul 16
AgentCompass: A Unified Evaluation Infrastructure for Agent Capabilities
arXiv:2607. 13705v1 Announce Type: new Abstract: As Large Language Models (LLMs) evolve into autonomous agents, the need for unified evaluation infrastructure becomes critical.
arXiv AI
Jun 6
Evaluating Agentic Configuration Repair for Computer Networks
arXiv:2606. 06212v1 Announce Type: new Abstract: Misconfigurations in computer networks remain a major source of critical Internet outages.
arXiv AI
2d ago
Benchmarking AI Agents for Hardware Design Automation via MCP Tool Calling
The paper investigates whether locally deployed large language models can automate hardware design workflows that involve repetitive, dependency-ordered operations using specialized tools. A Model Context Protocol (MCP) server is created to emulate a proprietary hardware design tool, and a benchmark tests single and multi-step edits, invalid requests, misspelled prompts, and multi-server contexts. Seven open-source models are evaluated across different pipeline choices, revealing that strong models can nearly fully cover expected calls, but reliability hinges on task structure and agent configuration, with comprehensive tool descriptions reducing failures and multi-agent setups aiding weaker models at the cost of extra calls.
arXiv AI
Jul 14
A Formal Hierarchical Architecture for Agentic Orchestration with Stack-Based Execution and Lazy Discovery
arXiv:2607. 11138v1 Announce Type: new Abstract: The rapid expansion of capabilities in Large Language Model (LLM) agents has exposed a critical architectural bottleneck: when agents are given access to a flat, monolithic registry of tools, the model must evaluate hundreds or thousands of options simultaneously.
arXiv AI
Aug 12
Conversational Orchestration for Organic 6G
arXiv:2608. 10714v1 Announce Type: cross Abstract: The Organic 6G vision of a network of networks spanning an edge-cloud continuum complemented by non-terrestrial resources requires, to realize its promise, service provisioning that is simple to operate, scalable across independently administered domains, and agile under domain churn (i.
arXiv Machine Learning
Jul 14
NetInjectBench: Benchmarking Indirect Prompt Injection in Tool-Using Large Language Model Agents for Network Operations
arXiv:2607. 10490v1 Announce Type: cross Abstract: Tool-using large language model (LLM) agents are attractive for network operations, but tickets, alerts, logs, runbooks, and ChatOps messages can carry indirect prompt injections.