arXiv:2506. 11066v3 Announce Type: replace-cross Abstract: Code retrieval is essential in modern software development, as it boosts code reuse and accelerates debugging.
By Jiahui Geng, Fengyu Cai, Shaobo Cui, Qing Li, Liangwei Chen, Chenyang Lyu, Haonan Li, Derui Zhu, Walter Pretschner, Heinz Koeppl, Fakhri Karray
arXiv:2606. 28998v1 Announce Type: cross Abstract: Large Language Model (LLM) alignment trains an LLM using preference data to produce outputs that better meet established quality standards.
By Gias Uddin, Sanjeepan Sivapiran
arXiv:2601. 02430v3 Announce Type: replace-cross Abstract: Web applications (web apps) have become a key arena for large language models (LLMs) to demonstrate their code generation capabilities and commercial potential.
By Chenxu Liu, Yingjie Fu, Wei Yang, Ying Zhang, Tao Xie
Litmus is a zero-label system that automatically designs evaluation and monitoring metrics for AI pipelines by extracting evaluation intent from source code and targeted interrogation. It identifies what must be measured and why, then generates a justified, per-stage metric portfolio. In experiments on three real code-defined pipelines, Litmus outperforms AutoMetrics and DynamicRubric baselines in concern coverage, stage span, redundancy, and validity against per-row quality labels, especially on scientific QA.
By Prajjwal Gupta, Prasang Gupta, Vishal Bhutani, Apoorva Sharma, Sumanth Chundru, Waqar Sarguroh, Kevin Paul
arXiv:2607. 18642v1 Announce Type: new Abstract: Mined code corpora are abundant but uncontrolled: a snippet's semantics, surface "messiness," and difficulty are whatever the wild contained; there is no known-optimal reference to grade against; and any public sample may already sit in a model's training set.
By Yuxiang Ji
The paper introduces SWE-Flux, a repository‑level benchmark designed to test large language models’ ability to reason about runtime behavior. It contains 480 execution‑grounded instances from 12 real Python repositories, with gold answers automatically harvested from instrumented test executions. Evaluation of five LLMs shows the task remains difficult, with the best model achieving only 37% accuracy, and the benchmark can generate challenging variants through input perturbation.
By Hamed Taherkhani, Mohammad Abdollahi, Melika Sepidband, Hridya Dhulipala, Tien N. Nguyen, Hadi Hemmati