arXiv AI

Evaluating Fine-Tuning and Metrics for Neural Decompilation of Dart AOT Binaries

arXiv:2607. 06125v1 Announce Type: cross Abstract: Neural decompilation is increasingly studied as a code-generation problem, yet its evaluation methodology remains underdeveloped for modern languages.

arXiv AI

Jun 30

Reward-Free Code Alignment from Pretrained or Fine-Tuned LLM: Unpacking the Trade-offs for Code Generation

arXiv:2606. 28998v1 Announce Type: cross Abstract: Large Language Model (LLM) alignment trains an LLM using preference data to produce outputs that better meet established quality standards.

By Gias Uddin, Sanjeepan Sivapiran

llms reinforcement-learning fine-tuning benchmarks safety

arXiv AI

Jun 2

BenchEvolver: Frontier Task Synthesis via Solution-Centric Evolution

arXiv:2606. 01286v1 Announce Type: cross Abstract: The rapid progress of frontier large language models has led to widespread benchmark saturation, limiting the ability of existing datasets to differentiate model capabilities or provide useful training signal.

By Yangzhen Wu, Aaron J. Li, Wenjie Ma, Li Cao, Ziheng Zhou, Mert Cemri, Shu Liu, Yuran Xiu, Chenxiao Yan, Haikun Zhao, Bin Yu, Ion Stoica, Dawn Song

llms reinforcement-learning benchmarks

arXiv AI

Aug 5

Instruction Stacking Collapse: A Benchmark and the Capability-Dependent Value of Prompt Compilation

arXiv:2608. 02639v1 Announce Type: cross Abstract: Production prompts rarely carry a single instruction.

By Atul Anand, Sourav Chattaraj

llms benchmarks

arXiv AI

Jun 2

How Generation Architecture Shapes Code Complexity in Multi-Agent LLM Systems: A Paired Study on HumanEval

arXiv:2606. 00308v1 Announce Type: cross Abstract: Large-language-model code generation has shifted from single-shot prompting to multi-agent orchestrations - analyst, coder, tester, and debugger pipelines - and is evaluated almost exclusively on functional correctness.

By Nazmus Ashrafi

llms agents benchmarks

arXiv Machine Learning

Jul 20

VarRate: Training-Free Variable-Rate KV Cache Compression for Long-Context LLMs

arXiv:2607. 15498v1 Announce Type: cross Abstract: The key-value (KV) cache is the main memory bottleneck in long-context large language model (LLM) inference.

By Shahrzad Esmat, Dhawal Shah, Ali Jannesari

llms efficiency benchmarks

arXiv AI

Aug 7

Recursive Synthesis for Long-Horizon Terminal Tasks

arXiv:2608. 05466v1 Announce Type: new Abstract: High-quality long-horizon training data for terminal agents is expensive to produce, often costing hundreds to thousands of dollars per task, because each task must keep the instruction, environment, reference solution, and verifier mutually consistent.

By Zhongzhi Li, Yucheng Shi, Zongxia Li, Ruhan Wang, Anhao Li, Zixun Huang, Junyao Yang, Lei Ke, Ninghao Liu, Haitao Mi, Leowei Liang

llms agents reinforcement-learning fine-tuning benchmarks

arXiv AI

Jun 9

Lost in the Flow with Code Talkers: Unveiling the Instruction-Tuning Tax of Large Language Models in Code Tasks

arXiv:2606. 08676v1 Announce Type: cross Abstract: AI coding assistants have significantly improved developer productivity by automatically suggesting code that aligns with user intent, and many of these tools are now integrated directly into Integrated Development Environments (IDEs).

By Shi Ying Chang, Chiok Yew Ho, Yichen Li, Yintong Huo

llms fine-tuning

arXiv Machine Learning

Jun 11

SwiftCTS: Fast Cross-Design Prediction and Pareto Optimization of Clock Tree Metrics via Few-Shot Calibration

arXiv:2606. 11348v1 Announce Type: new Abstract: Clock Tree Synthesis (CTS) is a computationally expensive stage in the physical design flow, requiring iterative EDA tool invocations to navigate a vast configuration space for optimal power, wirelength, and timing skew.

By Barsat Khadka, Kawsher Roxy, Md Rubel Ahmed

fine-tuning benchmarks

arXiv AI

Jun 11

Soft-Prompt Tuning for Fair and Efficient LLM Benchmark Evaluation

arXiv:2606. 12117v1 Announce Type: cross Abstract: Benchmark scores often misrepresent a large language model's (LLM's) knowledge, because they rely, e.

By Selen Erkan, Bastian Boll, Kristian Kersting, Bj\"orn Deiseroth, Letitia Parcalabescu

llms benchmarks

Hugging Face Trending Papers

Jun 24

Evaluating LLMs on Real-World Software Performance Optimization

Software performance optimization is a notoriously complex and manual task. Despite the growing use of Large Language Models (LLMs) for code refinement, we still lack benchmarks that capture how optimization actually happens in real-world codebases.

llms efficiency benchmarks

arXiv AI

Jul 24

ExecuGraph: A Multi-Agent, Execution-Grounded Framework for Reliable Backend Code Synthesis with Large Language Models

arXiv:2607. 20499v1 Announce Type: new Abstract: Large Language Models generate plausible backend code, but a single-pass paradigm provides no guarantee of correctness or runtime reliability.

By Sai Deekshith Lekkala, Jothi Prabha Appadurai, Rohith Reddy Bellibatlu, Manpreet Singh

llms agents benchmarks

arXiv Machine Learning

Jul 23

Building Fast, Evaluating Slow: Pipeline Choices Dominate Autointerpretability Score Variance

arXiv:2607. 19386v1 Announce Type: new Abstract: Cross-paper comparison of sparse autoencoder (SAE) interpretability often relies on autointerpretability scores.

By Sinie van der Ben, Neele Roch, Anna Hedstr\"om, Mennatallah El-Assady

llms safety

Related stories