The study investigates how natural-language comments influence code generation by large language models. Observational and controlled experiments on LiveCodeBench reveal that comment frequency and general intent do not predict success, but comments derived from correct solutions significantly improve recipient model performance by an average of 17.2%. Conversely, comments from failed solutions or unrelated problems either offer no benefit or even reduce performance, and most models cannot fully recover the advantage of well‑crafted comments.
Large Language Models (LLMs) often generate natural-language comments while writing code, and these comments become part of the context used to generate the code that follows. However, it remains uncl...
arXiv:2601. 19072v3 Announce Type: replace-cross Abstract: Large Language models (LLMs) have shown strong capabilities in code review automation, such as review comment generation, yet they suffer from hallucinations -- where the generated review comments are ungrounded in the actual code -- poses a significant challenge to the adoption of LLMs in code review workflows.
The use of LLMs in software development has become increasingly widespread on tasks such as code generation and summarization. Reports from large technology companies showed that around 20% to 30% of their code are generated by LLMs.
arXiv:2607. 01867v1 Announce Type: cross Abstract: The use of LLMs in software development has become increasingly widespread on tasks such as code generation and summarization.
arXiv:2501. 11086v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs) have shown tremendous promise in automated software engineering.
arXiv:2607. 10674v1 Announce Type: cross Abstract: As AI code tools become integrated into programming environments, students increasingly describe intended behavior in natural language and rely on these tools to generate code, shifting emphasis from code writing to specification.
arXiv:2606. 24267v2 Announce Type: replace-cross Abstract: While in-context learning is generally shown to be effective in Large Language Models (LLMs), bad contexts can cause performance degradation and mode collapse, a phenomenon we call "pigeonholing.
arXiv:2606. 00049v1 Announce Type: cross Abstract: Large language models (LLMs) are widely recognised for their applications in natural language generation and are increasingly used for code generation tasks.
Large Language Models (LLMs) perform differently on identical programming tasks when prompted in different natural languages, a phenomenon known as language bias. While this behavior has been widely studied for general text generation, its impact on code generation quality and programming conventions remains largely unexplored.
CriticGen introduces a generation‑aware evaluation framework that generates sample‑specific evaluation dimensions and scoring criteria across categories such as subjective, objective, and self‑derived constraints. These dynamic rubrics produce a score, reason, executable refinement suggestion, and a refined answer, enabling models to diagnose and target flaws in their responses. Experiments show significant gains in rubric quality, score correlation, and actionable feedback, with 73.17% of answers improved and a 93.28% non‑degradation rate.
arXiv:2607. 14816v1 Announce Type: cross Abstract: Large Language Models (LLMs) perform differently on identical programming tasks when prompted in different natural languages, a phenomenon known as language bias.
arXiv:2606. 24267v1 Announce Type: cross Abstract: While in-context learning is generally shown to be effective in Large Language Models (LLMs), bad contexts can cause performance degradation and mode collapse, a phenomenon we call "pigeonholing.