Microservice architectures have become dominant for modernizing monolithic systems, yet identifying appropriate services remains challenging and largely manual. Existing decomposition approaches are predominantly code-centric, limiting applicability in early design stages where only textual requirements are available.
arXiv:2606. 01385v1 Announce Type: cross Abstract: Software architecture design is a critical yet inherently complex and knowledge-intensive phase that requires balancing competing quality attributes and adapting to evolving requirements.
arXiv:2606. 29520v1 Announce Type: cross Abstract: Large Language Models (LLMs) are increasingly used as assistants across the software development lifecycle, yet their ability to reason about software architecture remains largely unmeasured.
arXiv:2606. 05720v1 Announce Type: cross Abstract: Large language models and AI coding agents have reshaped software development, but the path to fully AI-native systems faces structural challenges.
The study investigates whether large language models can extract Architectural Design Decisions (ADDs) from source code commits. Using four LLMs (Gemini 3 Pro, DeepSeek R1, Kimi K2, Qwen3) with zero‑shot and few‑shot prompting on 30 developer‑written ADDs, the authors evaluate outputs with ROUGE‑L, BLEU, METEOR, and BERTScore. Results show all models achieve a BERT‑F1 above 0.81, with few‑shot prompting slightly improving alignment, but the generated ADDs tend to be overly long, implementation‑focused, and lack the rationale behind the decisions.
Agent Seer is a pipeline that automatically synthesizes realistic evaluation scenarios for AI agents that use external tools, using only the tool’s specification (function names, natural‑language descriptions, and typed parameter schemas). Starting from a single Model Context Protocol (MCP) specification, it enriches raw schemas, generates graded scenarios with synthetic tool outputs, and expands them into mock‑data‑grounded multi‑turn dialogues that demonstrate strong tool‑calling correctness and conversational coherence. Across seven diverse MCP specifications, the pipeline achieves high quality, with parameter‑schema complexity emerging as the main driver of quality variation and argument‑value accuracy identified as the dominant failure mode.
arXiv:2604. 22207v2 Announce Type: replace-cross Abstract: Due to the textual and repetitive nature of many Requirements Engineering (RE) artefacts, Large Language Models (LLMs) have proven useful to automate their generation and processing.
This paper presents a black-box evaluation framework to systematically assess the ability of Large Language Models (LLMs) to generate Design Structure Matrices (DSMs) from structured technical documentation. Motivated by the closed-source nature of current Auto-DSM pipelines, the framework introduces a reproducible methodology that benchmarks generated DSMs (GEN-DSMs) against manually validated ground-truth matrices (GT-DSMs).
arXiv:2607. 05985v1 Announce Type: new Abstract: This paper presents a black-box evaluation framework to systematically assess the ability of Large Language Models (LLMs) to generate Design Structure Matrices (DSMs) from structured technical documentation.
arXiv:2607. 22759v1 Announce Type: cross Abstract: Large language models (LLMs) show promise in code generation, but their capabilities to produce correct, synthesizable hardware description language (HDL) code still remain to be properly benchmarked.
Intent Engine is a natural‑language intent translation architecture that converts user intents into validated Service‑level Objectives (SLOs) for compute‑continuum microservice placement. It combines schema‑constrained extraction, retrieval‑grounded value construction from monitored infrastructure, and validation against supported constraints to produce reliable SLO artifacts. In evaluations on a 716‑record dataset, Intent Engine outperformed prompting baselines and a rule‑based parser, achieving a 0.941 total F1 score with GPT‑4.1 mini and reducing downstream placement failures from 30.8% to 2.1%.
arXiv:2606. 14948v1 Announce Type: cross Abstract: LLMs have substantially improved software engineering yet real-world development requires architectural understanding.
The paper presents a unified evaluation of seven open reasoning language models across four benchmarks (ARC-Challenge, GSM8K, MATH levels 1–3, and TruthfulQA MC1) using a consistent 238-example subset and three prompting strategies (zero-shot, chain-of-thought, few-shot CoT). It reports not only accuracy but also Wilson confidence intervals, latency, VRAM usage, weighted aggregate performance, Pareto-efficient points, prompt-sensitivity, and compatibility diagnostics, revealing that Gemma-4-26B-A4B tops the weighted score while Gemma-4-E4B offers a strong practical trade-off. The study emphasizes that model rankings shift with prompting strategy and that deployment trade-offs remain crucial, advocating for a deployment-aware, multi-objective evaluation framework rather than a single-score leaderboard.