The paper introduces RobustTests, a framework that improves reinforcement learning for code generation by synthesizing test cases from faulty code and refining rewards with a dense, stepwise function. It uses validator agents and behavioral clustering to filter out invalid or redundant tests, and incorporates pass‑rate‑based rewards to counter hallucination noise. Experiments on CodeContests and LiveCodeBench show that fine‑tuning Qwen3‑32B with RobustTests yields a 3% absolute performance gain over baseline methods.
arXiv:2608. 16742v1 Announce Type: cross Abstract: Large Language Models (LLMs) have achieved remarkable progress in code generation, yet ensuring correctness in complex, repository-level tasks remains challenging.
arXiv:2509. 24148v3 Announce Type: replace-cross Abstract: Test-Driven Development (TDD) is a widely adopted practice that requires developers to create and execute tests alongside implementation.
Large Language Models (LLMs) have achieved remarkable progress in code generation, yet ensuring correctness in complex, repository-level tasks remains challenging. Existing approaches often use generated tests as static post-hoc validators, which limits their ability to guide implementation and may introduce misleading feedback when the tests themselves are incomplete or incorrect.
arXiv:2507. 22580v2 Announce Type: replace-cross Abstract: Automated Program Repair (APR) seeks to automatically correct software bugs without requiring human intervention.
arXiv:2602. 20213v2 Announce Type: replace-cross Abstract: The evaluation of Large Language Models (LLMs) for code generation relies heavily on the quality and robustness of test cases.
arXiv:2606. 12864v1 Announce Type: cross Abstract: Despite strong performance in competitive programming, the role of Large Language Models (LLMs) in supporting human learning in the same setting remains largely unexplored.
arXiv:2505. 13553v3 Announce Type: replace-cross Abstract: The hallucination of code generation models hinders their applicability to systems requiring higher safety standards.
arXiv:2607. 02057v1 Announce Type: cross Abstract: In recent years, it has become increasingly evident that large language models (LLMs) and autonomous agents raise the level of abstraction in software development by shifting the focus from writing precise procedures to expressing intents and goals.
arXiv:2607. 22880v1 Announce Type: cross Abstract: Recent advances in large language models (LLMs) have driven growing interest in using LLMs to automate test generation.
While Large Language Models (LLMs) show great promise for automating unit test generation, recent studies suggest that the quality of generated tests can be negatively impacted when models are prompted with buggy code. This paper presents a new metric to quantitatively measure the "misguidance effect," a phenomenon where buggy code steers LLMs toward generating tests that validate its erroneous behavior rather than expose it.
arXiv:2607. 11342v1 Announce Type: cross Abstract: Despite their central role in fault detection, test oracles remain challenging to construct effectively.
The paper introduces CodeRQ-Bench, the first benchmark for assessing large language model reasoning quality across coding tasks such as generation, summarization, and classification. It analyzes over a thousand mismatches from existing evaluators, identifies recurring limitations, and derives design insights that lead to a new two‑stage evaluator, VERA. Experiments show VERA outperforms strong baselines, improving AUCROC by up to 0.26 and AUPRC by up to 0.21 on four datasets.