arXiv AI
Aug 18
Automating the Detection of Requirement Dependencies Using Large Language Models
arXiv:2602. 22456v2 Announce Type: replace-cross Abstract: Requirements are inherently interconnected through various types of dependencies.
Cluster-Aware Dual-Level Test Specification Generation for Large-Scale Automotive Software Requirements
arXiv:2606. 17197v1 Announce Type: cross Abstract: Generating test specifications that satisfy Automotive SPICE SWE.
Related stories
arXiv AI
Jun 8
AI-Driven Test Case Generation from Natural Language Requirements: A Survey of Techniques and Research Gaps
arXiv:2606. 06563v1 Announce Type: cross Abstract: Software testing is critical for verifying that systems meet specified requirements, yet remains among the most time-consuming and expensive activities in development.
arXiv AI
Jul 21
Integrating High-Level Requirements to Low-Level Tests with Machine-Readable V&V Specifications
arXiv:2607. 17686v1 Announce Type: cross Abstract: Modern software teams have mature tools for low-level testing, such as pytest, JUnit, and Jest, which make it inexpensive to write unit tests and run them on every commit.
arXiv AI
Aug 17
RepBench: Compiling Benchmarks into Capability Representations for Large Language Models
arXiv:2607. 28008v2 Announce Type: replace-cross Abstract: Representation engineering reads and steers capability directions in large language models, yet methods are typically evaluated on paper-specific synthetic data.
arXiv AI
Jul 8
Auto-DSM Under the Lens: A Black-Box Evaluation Framework for LLM-Based DSM Generation
arXiv:2607. 05985v1 Announce Type: new Abstract: This paper presents a black-box evaluation framework to systematically assess the ability of Large Language Models (LLMs) to generate Design Structure Matrices (DSMs) from structured technical documentation.
arXiv AI
Jul 21
Automated Hardware Validation Test Plan Generation for Large Scale AI Datacenter Platforms Using a Generative AI Multi-Agents Architecture
arXiv:2607. 16388v1 Announce Type: cross Abstract: Large-scale AI datacenter platforms comprise thousands of heterogeneous hardware components whose validation requires comprehensive fault injection test plans.
arXiv AI
Aug 5
Evaluating LLM-Based Goal Extraction in Requirements Engineering: Prompting Strategies and Their Limitations
arXiv:2604. 22207v2 Announce Type: replace-cross Abstract: Due to the textual and repetitive nature of many Requirements Engineering (RE) artefacts, Large Language Models (LLMs) have proven useful to automate their generation and processing.
Hugging Face Trending Papers
Jul 7
Auto-DSM Under the Lens: A Black-Box Evaluation Framework for LLM-Based DSM Generation
This paper presents a black-box evaluation framework to systematically assess the ability of Large Language Models (LLMs) to generate Design Structure Matrices (DSMs) from structured technical documentation. Motivated by the closed-source nature of current Auto-DSM pipelines, the framework introduces a reproducible methodology that benchmarks generated DSMs (GEN-DSMs) against manually validated ground-truth matrices (GT-DSMs).
arXiv AI
Jul 7
A Retrieval-Augmented Framework for Detecting and Resolving Pragmatic Ambiguities in Natural Language Requirements
arXiv:2607. 04436v1 Announce Type: cross Abstract: Natural language requirements (NLRs) are essential for bridging communication gaps among diverse stakeholders in software development.
arXiv AI
Jul 20
CRAFT: Clustering Rubrics to Diagnose Weak LLM Capabilities and Generate Targeted Fine-Tuning Data
arXiv:2607. 16122v1 Announce Type: new Abstract: Evaluations should do more than measure a models current performance.
arXiv AI
Jul 17
OmniaBench: Benchmarking General AI Agents Across Diverse Scenarios
arXiv:2607. 14989v1 Announce Type: cross Abstract: Large language models are increasingly evolving from text generators into general agents capable of understanding user requests, invoking external tools, and completing complex tasks through interaction.
arXiv Computation and Language
Sep 22
CCTU: A Benchmark for Tool Use under Complex Constraints
CCTU is a new benchmark designed to evaluate large language models (LLMs) on their ability to use tools under complex constraints. It includes 200 test cases that average seven constraint types and 4,700‑token prompts, covering resource, behavior, toolset, and response dimensions. An executable validation module performs step‑level checks, and nine state‑of‑the‑art LLMs were tested, revealing that none exceed a 20% task completion rate when strict constraints are enforced, with frequent violations and limited self‑refinement.