arXiv AI
An Empirical Evaluation of Cost-Efficient Large Language Models on Algorithmic Programming Tasks
Read the original on arXiv AI →The paper evaluates three cost‑efficient large language models—Gemini Flash 3, GPT‑5.4 mini, and Claude Haiku 4.5—on 992 algorithmic programming tasks written as Java Spring Boot service methods. Across eight configurations (model, coding tool, and prompt variants) the authors generated 7,593 methods, classified them into eight outcome categories, and executed 7,936 requests. While structural compliance was high, only 12.9% of returned answers were correct, and 38.4% of methods failed to compute the value they returned, with the most reliable methods being the least accurate.
Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.