arXiv AI
Revision-Based Learning from Interactive Experience and Feedback
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv Machine Learning
Jun 11
FronTalk: Benchmarking Front-End Development as Conversational Code Generation with Multi-Modal Feedback
arXiv:2601. 04203v2 Announce Type: replace-cross Abstract: We present FronTalk, a benchmark for front-end code generation that pioneers the study of a unique interaction dynamic: conversational code generation with multi-modal feedback.
Hugging Face Trending Papers
3d ago
Catalyzing Interactive HTML Generation Capabilities via High-Fidelity Reinforcement Learning
Generating HTML from natural-language specifications is an important application of code large language models (LLMs). Despite producing renderable pages, current models often fail to ensure functiona...
arXiv AI
Aug 12
Do Personalized Skills Help Coding Agents? An Empirical Study of Developer Interaction Histories
arXiv:2608. 10319v1 Announce Type: cross Abstract: Large language model (LLM)-powered agents have rapidly evolved from code-completion tools into solvers of complex software engineering tasks.
arXiv AI
6d ago
Do Personalized Skills Help Coding Agents? An Empirical Study of Developer Interaction Traces
The paper investigates whether tailoring coding agent skills to individual developers improves collaboration. It introduces a framework that bootstraps rule‑based skills and refines them with LLMs to capture developer‑specific preferences, and evaluates this approach using an interactive replay system with a simulated developer. Experiments on real interaction logs show that while task‑specific skills generally perform best, personalized skills become more effective when developers frequently express consistent preferences, especially on related tasks.
arXiv Computation and Language
Sep 3
User Feedback Provides a Unique Signal that LLMs Can not Detect
The paper argues that user feedback from real interactions is a valuable learning signal for Large Language Models (LLMs), contrary to recent claims that it is too noisy to use. By creating synthetic data with a clear ground truth and testing on naturalistic data, the authors show that revisions guided by user feedback fix targeted issues more often than baseline revisions. They further reveal that current evaluation methods bias against feedback‑driven improvements, as judges tend to overlook genuinely corrected responses and favor inferior baselines.
arXiv AI
Oct 9
SWE-Journey: Towards More Realistic Evaluation of Coding Assistants through Long-Horizon, Multi-Turn Interaction
SWE-Journey is a new benchmark designed to evaluate coding assistants in more realistic settings, focusing on long-horizon tasks and multi-turn interactions. It introduces a weak-to-strong synthesis pipeline to create extended coding challenges and a user-simulation agent based on four real user personas to mimic authentic code-assistance dialogues. The benchmark reveals that while models succeed on over 75% of tests with software architects, they manage less than 25% with non-coders, highlighting a significant gap in current assistant capabilities.