A research agenda for assessing the economic impacts of code generation models

BigCodeArena: Judging code generations end to end with code executions

Measuring and Mitigating Bias in Code Generated by Large Language Models

arXiv:2606. 00049v1 Announce Type: cross Abstract: Large language models (LLMs) are widely recognised for their applications in natural language generation and are increasingly used for code generation tasks.

A hazard analysis framework for code synthesis large language models

Beyond Code Generation: Reliability, Verification, and Cost Economics in the Agentic Software Development Lifecycle

The paper examines how AI coding agents are evolving beyond simple autocomplete to perform complex tasks such as repository inspection, multi-file editing, tool execution, test writing, pull request creation, and long-duration work with minimal supervision. It highlights that while these agents boost coding activity, significant bottlenecks remain in review, integration, testing, security, deployment, and production operations, and that the economics of software development are shifting toward variable token, tool, sandbox, CI, and rework costs. The authors synthesize recent research and industry data to propose four engineering concepts—Agentic SDLC Throughput Paradox, Production-Qualified Change, Verification Tax, and an Agentic SDLC Control Plane—to guide the allocation of autonomy within cost, reliability, and human-attention constraints, ultimately reframing the research focus to production-qualified value per dollar, reviewer-hour, and operational risk.

Towards Functional Correctness of Large Code Models with Selective Generation

arXiv:2505. 13553v3 Announce Type: replace-cross Abstract: The hallucination of code generation models hinders their applicability to systems requiring higher safety standards.

When Who You Are Can Change the Code You Get: A Study of Persona-Induced Bias in LLM Code Generation

arXiv:2609.22102v1 Announce Type: cross Abstract: Large Language Models (LLMs) are widely used as programming assistants, yet it remains unclear whether and how user's demographic information impacts...

An Exploratory Study on LLM-Generated Code and Comments in Code Repositories

The use of LLMs in software development has become increasingly widespread on tasks such as code generation and summarization. Reports from large technology companies showed that around 20% to 30% of their code are generated by LLMs.

Cheap Code, Costly Judgment: A Case Study on Governable Agentic Software Engineering

arXiv:2607. 01087v1 Announce Type: cross Abstract: Generative AI is shifting software engineering from a practice organized around scarce implementation effort toward one organized around abundant, low-cost code production.

An Exploratory Study on LLM-Generated Code and Comments in Code Repositories

arXiv:2607. 01867v1 Announce Type: cross Abstract: The use of LLMs in software development has become increasingly widespread on tasks such as code generation and summarization.

Adoption and Impact of Command-Line AI Coding Agents: A Study of Microsoft's Early 2026 Rollout of Claude Code and GitHub Copilot CLI

arXiv:2607. 01418v1 Announce Type: cross Abstract: Organizations rolling out agentic command line tools like Anthropic's Claude Code and GitHub's Copilot CLI need to know who will try them, who will keep using them, and whether the tools produce enough output to justify their cost.

CIDR: A Large-Scale Industrial Source Code Dataset for Software Engineering Research

arXiv:2605. 12153v2 Announce Type: replace-cross Abstract: We present the Curated Industrial Developer Repository (CIDR), a large-scale dataset of real-world software repositories collected from industrial partners.

CS-Guard: Benchmarking LLM Guardrails for Code Generation Security

CS-Guard is a new benchmark that systematically evaluates guardrails for code generation security, covering 1,000 malware-generation prompts, 7 jailbreak attacks, and a novel fictional scenario attack (FSA) for text-to-code generation, as well as 331 code prompts for code-to-code generation. The study empirically tests nine guardrails across seven large language models, finding that many guardrails fail to prevent malicious code generation, with attack success rates reaching about 50% for text-to-code and up to nearly 100% for code-to-code and FSA scenarios. CS-Guard introduces a modular three-layer guardrail taxonomy and releases its benchmark and data to support future research.