The study investigates whether large language models can extract Architectural Design Decisions (ADDs) from source code commits. Using four LLMs (Gemini 3 Pro, DeepSeek R1, Kimi K2, Qwen3) with zero‑shot and few‑shot prompting on 30 developer‑written ADDs, the authors evaluate outputs with ROUGE‑L, BLEU, METEOR, and BERTScore. Results show all models achieve a BERT‑F1 above 0.81, with few‑shot prompting slightly improving alignment, but the generated ADDs tend to be overly long, implementation‑focused, and lack the rationale behind the decisions.
By Amey Karan, Rudra Dhar, Mohamed Soliman, Karthik Vaidhyanathan
Spec-Driven Development (SDD) is a practice where developers and AI coding agents write specifications to guide software implementation, aiming to preserve the intent behind natural‑language instructions. This study analyzes SDD through Reddit discussions, interviews with 21 practitioners (including eight tool makers), and a large corpus of 470,795 spec files from 73,030 GitHub repositories. Findings reveal that while specs often include implementation details that aid agents, they also risk decay; tool-specific structures influence perception, and benefits vary between individuals and teams, with SDD mainly used for new features and involving higher planning and review costs.
By Shyam Agarwal, Anmol Singhal, Andrew Macvean, Bogdan Vasilescu, Travis Breaux
The study investigates how users interact with ChatGPT for code generation beyond simple function-level tasks, focusing on project-level benchmarks that involve multi-class dependencies. A user study with 36 participants examined prompting patterns, screen recordings, and chat logs to identify Human‑LLM Interaction (HLI) features that influence productivity. The results highlight three consistently supportive HLI features, five guidelines to boost productivity, and a taxonomy of 29 runtime and logic errors with mitigation strategies.
By Sangwon Hyun, Hyunjun Kim, Jinhyuk Jang, Hyojin Choi, M. Ali Babar
arXiv:2609.22102v1 Announce Type: cross
Abstract: Large Language Models (LLMs) are widely used as programming assistants, yet it remains unclear whether and how user's demographic information impacts...
By Anubhav Gupta, Mayara Costa Figueiredo, Leticia Santos Machado, Tanner Wright, Ivan Beschastnikh, Cleidson R. B. de Souza, Gema Rodr\'iguez-P\'erez
The use of LLMs in software development has become increasingly widespread on tasks such as code generation and summarization. Reports from large technology companies showed that around 20% to 30% of their code are generated by LLMs.
arXiv:2607. 01867v1 Announce Type: cross Abstract: The use of LLMs in software development has become increasingly widespread on tasks such as code generation and summarization.
By Yongyi Ji, Jiaji Wang, Yi Zhou, Fuxiang Chen, Hongji Yang
The paper "LLM Code Smells: A Taxonomy and Detection Approach" presents a refined taxonomy of nine LLM code smells and introduces SpecDetect4LLM, a static analysis tool for detecting them. Extensive empirical evaluation across 692 open‑source projects shows that 73.5% of systems contain LLM code smells, with the tool achieving 91.3% precision and 71.8% recall.
By Zacharie Chenail-Larcher, Brahim Mahmoudi, Naouel Moha, Quentin Sti\'evenart, Florent Avellaneda
arXiv:2608. 11513v1 Announce Type: cross Abstract: Large Language Models (LLMs) are increasingly integrated into software engineering workflows, helping developers write, debug, test, and maintain code.
By Alex Deaconu, Anubhav Gupta, Manaal Basha, Nicholas Haydu, Gema Rodr\'iguez-P\'erez
The paper introduces ReqClarifyBench, an interactive benchmark for evaluating how well large language models (LLMs) can clarify vague user intent into clear requirements in repository-level software development. It includes 341 tasks across game, miniapp, and web development, each with a vague intent, a human-validated requirement document, and code evaluation artifacts, and uses LLM-based user simulators with diverse personas. The study evaluates six LLMs on clarification effectiveness and requirement quality, finding that stronger clarification does not always yield better documents, performance varies by persona, and coding-agent harnesses do not improve document quality over base LLMs.
By Mingyu Chen, Zhiqing Zhong, Manyi Wang, Linxi Liang, Rouyu Chen, Pinjia He
The article presents GUEST, a framework of 51 recommendations for using and evaluating generative AI (GenAI) tools in systematic literature reviews (SLRs). It identifies four methodological categories—SLR practice adjustments, secondary‑study standards, GenAI evaluation practices, and core empirical rigour—to guide researchers in planning, executing, and reporting GenAI‑assisted reviews. The authors conclude that while GenAI cannot replace human oversight for unsupervised SLRs, it can efficiently support repetitive screening and secondary validation tasks.
By Barbara Kitchenham, Sebasti\'an Pizard, Lech Madeyski, Ronnie de Souza Santos, Martin Shepperd, David Budgen
The study evaluates Vibe Coding, an AI‑led conversational programming paradigm that lets developers generate software via natural‑language interaction with large language models. In a mixed‑methods experiment with 30 participants, Vibe Coding improved development efficiency—reducing task completion time by 27% versus traditional coding and 12% versus AI‑assisted coding—while also yielding a good usability score (SUS = 71.4) and moderate cognitive workload (NASA‑TLX = 55.5). However, the gains came with trade‑offs: lower maintainability indices, higher security vulnerabilities, and themes of trust calibration, loss of control, and prompt‑engineering strategy emerged, leading the authors to propose a three‑pillar framework for responsible adoption.
By Sales G. Aribe Jr., Louie Jay S. Labastida
arXiv:2608.30756v1 Announce Type: cross
Abstract: Large language models (LLMs) have become an essential tool for assisting developers, yet we still lack knowledge on ways to effectively support their...
By Annemarie Wittig, Alina Mailach, Janet Siegmund, Norbert Siegmund