← Back to all news
arXiv AI

Quality, Not Just Outcome: A Structural Scoring Framework for Selecting Coding-Agent Trajectories for Supervised Fine-Tuning

The paper introduces TQS, a Trajectory Quality Score that evaluates coding-agent interaction trajectories based on structural signals such as information focus, intent–action alignment, and policy dynamics, rather than just final outcomes. TQS operates in linear time without requiring an LLM judge and is applied to 155,144 successful trajectories from a public corpus. Fine‑tuning agents on trajectories ranked highest by TQS consistently outperforms models trained on random or low‑ranked subsets on the SWE‑Bench‑Verified benchmark, demonstrating the value of process‑level quality filtering.

Related stories

arXiv AI
Aug 28

SWE-Prime: Fewer Trajectories, Better Performance

SWE-Prime introduces a two-stage supervised fine-tuning data selection process for large language models tackling software issues. The first stage filters entire trajectories by quality and representativeness, while the second stage selects meaningful semantic segments based on contribution, learnability, and risk. Experiments on SWE-Bench Pro and Verified demonstrate that training on just 10% of trajectories chosen by SWE-Prime surpasses full-dataset training, achieving up to 12.2% and 24.2% performance gains.

By Dewu Zheng, Ruizhe Ye, Yanlin Wang, Yang Ye, Hongyu Zhang, Ensheng Shi, Xilin Liu, Yuchi Ma, Jianxing Yu, Zibin Zheng
llmsagentsfine-tuning
More like this →
arXiv Machine Learning
Oct 19

MetaCurate: Evolving Curation Filters for High-Quality Agentic Coding Data

MetaCurate is a training‑free framework that automatically evolves curation filters for agentic coding data by using pairwise large language model judgments to refine its rules, rubric, and aggregation policy. Starting from no initial filter, it discovers a strategy that scores trajectories independently, requiring pairwise comparisons only during the search phase. Experiments on an uncurated trajectory pool and the Nemotron v3 SWE dataset show that MetaCurate achieves the highest mean SWE‑Verified score compared to random selection, hand‑authored filtering, and a training‑loop baseline, while incurring lower search costs.

By Donghoon Kim, Seunghyuk Oh, Kevin Galim, Minseo Kim, Minjae Lee, Younghyun Kim, Wooseok Seo, Hyung Il Koo, Wonjun Kang
llmsagentsfine-tuning
More like this →
Hugging Face Trending Papers
Aug 27

SWE-Prime: Fewer Trajectories, Better Performance

SWE-Prime introduces a two‑stage, multi‑granularity supervised fine‑tuning (SFT) data selection process for large language models tackling real‑world software problems. The first stage filters entire trajectories based on process quality, result quality, and representativeness, while the second stage evaluates semantic segments for contribution, learnability, and risk, keeping all segments in context but only penalizing selected ones during training. Experiments on SWE‑Bench Pro and Verified demonstrate that training on just 10% of trajectories chosen by SWE‑Prime outperforms full‑dataset training, achieving up to 12.2% and 24.2% relative gains.

llmsagentsfine-tuning
More like this →