← Back to all news
arXiv AI

Agent or Human? Detecting Commits by AI Coding Agents in the Absence of Authorship Signatures

The study examines whether commits made by AI coding agents can be distinguished from human commits when authorship signatures are removed. Using nearly 8 million commits from over a thousand repositories, the authors find that detectors can achieve a ROC AUC of up to 0.859, largely driven by message verbosity and change size rather than code content. However, precision remains low (≈0.332) and detection performance drops on held‑out agents, indicating limited reliability without explicit attribution.

Related stories

Hugging Face Trending Papers
Jun 23

Detecting AI Coding Agents in Open Source: A Validated Multi-Method Census of 180 Million Repositories

Generative AI coding agents are entering the open-source supply chain, yet their diverse and often invisible traces leave their prevalence poorly understood. We introduce a multi-layered detection framework that integrates configuration-file scanning, commit-message analysis, author-identity matching, and bot-signature lookup across World of Code (180M+ Git repositories), classifying agent traces into four behavioral types.

llmsagents
More like this →
arXiv Machine Learning
Oct 22

What Git Does Not Record: Provenance Legibility in Two AI-Assisted Research Repositories

The paper audits two single‑investigator AI‑assisted research repositories to assess how well they record provenance of AI tool use. One repository contains a prescriptive trace of rule files, with 85.6% of committed lines still visible, while the other holds a narrative trace of an agent’s scratchpad, showing only 21.8% visibility. The study finds that repository‑level indicators such as the Co‑Authored‑By trailer are inconsistent and unreliable for measuring AI involvement, and it outlines which provenance claims each trace type can support or fail to support.

By Koutian Wu, Zesen Huang, Hejia Geng
agents
More like this →
arXiv AI
Sep 1

CrossAudit: A Git-Native, Cross-Vendor Audit Loop for Agentic Science

CrossAudit proposes a Git‑native, cross‑vendor audit protocol for autonomous research pipelines, ensuring each work increment is reviewed by an agent from a different vendor against a human‑written rulebook. Audit outcomes, disputes, and rulings are stored as git commits, providing a replayable, versioned supervision history. The authors implemented the protocol with GitHub Actions and Python, deployed it in a computational‑chemistry pipeline, and conducted a seeded‑defect trial that revealed differing interpretations of the same rulebook by two vendors.

By Zhaohe Dong, Yuhao Chen
agents
More like this →
arXiv AI
Oct 9

Runnable Commit Untangling for Coding Agents

The paper "Runnable Commit Untangling for Coding Agents" introduces RucTangle, an agentic method that untangles large, tangled patches from coding agents while ensuring each commit keeps the code runnable. It also presents TangleEval, an evaluation framework that measures how untangled commit histories aid agents in repairing bugs. Experiments on 131 agent-generated patches show RucTangle produces fully runnable histories, outperforming baselines that leave 20.6%-37.4% of commits unrunnable, and improves bug‑repair success by 5.2% absolute in pass@1.

By Jinfeng Jiang, Dongsun Kim, Dayi Lin, Zhou Yang
agents
More like this →
arXiv AI
5d ago

What Open-Source Projects Tell Their Coding Agents: A Census of AGENTS.md and CLAUDE.md

The study surveys popular, active GitHub repositories across thirteen languages, treating each repository’s AGENTS.md and CLAUDE.md files as instruction sets for coding agents. It finds that most repositories include such instructions, with CLAUDE.md often referencing AGENTS.md, and that these files contain a mix of project knowledge and workflow rules, including varying stances on AI attribution. The analysis reveals that more popular projects tend to have stricter pull‑request rules and AI contribution guidelines, and that LLM and agent projects are over‑represented among adopters.

By Pawel Zarzycki
llmsagents
More like this →