Can Coding Agents Reproduce Official Statistics? Metadata, Retry Budget and the Limits of Execution Feedback in a Controlled Eurostat Benchmark

The paper investigates whether large language models can reliably reproduce official Eurostat statistics by generating executable code. It evaluates a coding agent across four experimental conditions—task only, task plus metadata, metadata with a repair loop using execution feedback, and metadata with a retry budget but no diagnostics—using 30 natural‑language tasks spanning seven domains and datasets. Results show that success depends on semantic validation against frozen specifications, a fully specified output contract, and a retry budget, rather than on execution diagnostics alone.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

• llms
• agents
• benchmarks