arXiv AI
Do Not Copy/Paste: Soft Barriers for Copying in AI-Assisted Programming
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Jul 7
Refused in Chat, Written in Code: Workflow-Level Jailbreak Construction in IDE Coding Agents
arXiv:2607. 03968v1 Announce Type: cross Abstract: Large language models are increasingly deployed as IDE-integrated coding agents that decompose tasks, generate and edit files, run code, and refine outputs over many turns.
arXiv AI
Jun 9
Lost in the Flow with Code Talkers: Unveiling the Instruction-Tuning Tax of Large Language Models in Code Tasks
arXiv:2606. 08676v1 Announce Type: cross Abstract: AI coding assistants have significantly improved developer productivity by automatically suggesting code that aligns with user intent, and many of these tools are now integrated directly into Integrated Development Environments (IDEs).
arXiv AI
Aug 10
Characterizing the Quality Profile of AI-Generated C++ in Production
arXiv:2608. 06640v1 Announce Type: cross Abstract: The widespread integration of AI coding assistants offers undeniable boosts to engineering velocity.
arXiv AI
Oct 1
EngramBench: A Capability-Grounded Benchmark for Skill-Evolution Harnesses
EngramBench is a new benchmark designed to evaluate skill evolution in autonomous agents by focusing on genuine capability abstraction rather than solution copying. It includes 30 learning tasks and 13 unseen transfer tasks that require agents to manage complex, multi-hour development cycles with LLM‑simulated users. The study shows that while static skill banks cannot eliminate the need for precise code implementation, they effectively reduce redundant context and cut overall coding time by more than 55%.
arXiv AI
Oct 2
Are AI Coders Snitches? An Empirical Study of Pretraining Data Detection on Code Large Language Models
The paper investigates whether code large language models (CodeLLMs) inadvertently reproduce proprietary or sensitive code by evaluating seven state‑of‑the‑art training data detection (TDD) methods on eight CodeLLMs. It introduces CodeSnitch, a benchmark of 9,000 function‑level code samples across three languages, each labeled as included or excluded from training data, and applies mutation strategies based on the Type‑1 to Type‑4 code clone taxonomy to test TDD robustness. The study offers a systematic assessment of current TDD techniques for code and suggests directions for developing more effective detection methods.
arXiv AI
Oct 9
SWE-Journey: Towards More Realistic Evaluation of Coding Assistants through Long-Horizon, Multi-Turn Interaction
SWE-Journey is a new benchmark designed to evaluate coding assistants in more realistic settings, focusing on long-horizon tasks and multi-turn interactions. It introduces a weak-to-strong synthesis pipeline to create extended coding challenges and a user-simulation agent based on four real user personas to mimic authentic code-assistance dialogues. The benchmark reveals that while models succeed on over 75% of tests with software architects, they manage less than 25% with non-coders, highlighting a significant gap in current assistant capabilities.