arXiv AI June 2, 2026 By Yafan Huang, Sheng Di, Guanpeng Li

Not All Errors Are Equal: A Systematic Study of Error Propagation in Large Language Model Inference

arXiv:2606. 02430v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly integrated into high-performance computing (HPC) workflows, accelerating scientific discovery through diverse perspectives such as code generation and domain-specific decision-making.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv AI.

llms

arXiv AI

Jul 2

LLVM-Bench: Benchmarking and Advancing Large Language Models for LLVM Compiler Issue Resolution

arXiv:2607. 00700v1 Announce Type: cross Abstract: LLVM is a widely used compiler infrastructure whose scale and complexity make issue resolution labor-intensive and challenging.

By Zhao Tian, Yingquan Zhao, Chenyao Suo, Meng Wang, Junjie Chen

llms agents benchmarks

arXiv AI

Jul 14

SWE-MERA: A Dynamic Benchmark for Agenticly Evaluating Large Language Models on Software Engineering Tasks

arXiv:2507. 11059v3 Announce Type: replace-cross Abstract: The rapid advancement of Large Language Models (LLMs) in software engineering has revealed critical limitations in existing benchmarks, particularly the widely used SWE-bench dataset.

By Pavel Adamenko, Mikhail Ivanov, Aidar Valeev, Rodion Levichev, Pavel Zadorozhny, Ivan Lopatin, Dmitry Babaev, Alena Fenogenova, Valentin Malykh

llms agents benchmarks

arXiv AI

Aug 11

PluginEval: A Diagnostic Benchmark for Fine-Grained Error Attribution in Function Calling

arXiv:2608. 08700v1 Announce Type: new Abstract: Reliable evaluation of tool routing is critical as Large Language Models increasingly operate as autonomous agents.

By Dongjie Xu, Julius, Hanchi Dong, Minghua Tang, Yuxuan Sun, Ziwei Nie, Zicheng Liu, Dujun Qing, Jiajie Xu

llms agents benchmarks safety

arXiv AI

6d ago

Program Semantic Inequivalence Game with Large Language Models

arXiv:2505. 03818v3 Announce Type: replace-cross Abstract: Large Language Models (LLMs) can achieve strong performance on everyday coding tasks, but they can fail on complex tasks that require non-trivial reasoning about program semantics.

By Antonio Valerio Miceli-Barone, Vaishak Belle, Ali Payani

llms agents fine-tuning benchmarks

arXiv AI

Jun 24

VeriPilot: An LLM-Powered Verilog Debugging Framework

arXiv:2606. 23759v1 Announce Type: cross Abstract: Verilog debugging remains one of the most time-consuming stages in digital circuit design.

By Yihan Wang, Cheng Liu, Jiazheng Zhang, Lei Zhang, Long Cheng, Xiaowei Li, Huawei Li

llms benchmarks

arXiv AI

Jun 30

Diff-Based Code Corruption using LLMs for Large-Scale Bugfix Benchmarking

arXiv:2606. 29088v1 Announce Type: cross Abstract: There are various benchmarks to evaluate bugfixing capabilities of Large Language Models.

By Bal\'azs Szalontai, \'Abel Szauter, Bal\'azs M\'arton, P\'eter Verebics, Bal\'azs Pint\'er, Tibor Gregorics

llms fine-tuning benchmarks

Related stories

LLVM-Bench: Benchmarking and Advancing Large Language Models for LLVM Compiler Issue Resolution

SWE-MERA: A Dynamic Benchmark for Agenticly Evaluating Large Language Models on Software Engineering Tasks

PluginEval: A Diagnostic Benchmark for Fine-Grained Error Attribution in Function Calling

Program Semantic Inequivalence Game with Large Language Models

VeriPilot: An LLM-Powered Verilog Debugging Framework

Diff-Based Code Corruption using LLMs for Large-Scale Bugfix Benchmarking