arXiv AI

Which Algorithm Specification Formats Help Language Models Implement Machine Learning Algorithms?

arXiv:2607. 03158v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used to implement algorithms from research manuscripts, but papers often leave implementation choices implicit.

arXiv Machine Learning

Aug 3

Simulation Code Generation for Fluid Systems using Large Language Models: Benchmarking Models and Prompting Strategies

arXiv:2607. 29389v1 Announce Type: new Abstract: Large language models (LLMs) have demonstrated a strong ability to generate syntactically correct code from natural-language specifications.

By Jan Marius St\"urmer, Jascha Knack, Tobias Koch, Andreas Weinmann

llms benchmarks

arXiv Machine Learning

Jun 5

Can LLMs Write Correct TLA+ Specifications? Evaluating Natural-Language-to-TLA+ Generation

arXiv:2606. 05792v1 Announce Type: cross Abstract: TLA+ has supported industrial verification at companies such as Amazon and Microsoft, yet writing correct TLA+ specifications from natural language still requires time and expertise, which limits adoption.

By Arslan Bisharat, Brian Ortiz, Eric Spencer, Khushboo Bhadauria, TaiNing Wang, George K. Thiruvathukal, Konstantin Laufer, Mohammed Abuhamad

llms safety

arXiv AI

2d ago

Does ISO-Grounded NFR Specification Improve LLM Code Generation? A Comparison of Rich and Structured Interventions against a Natural-Language Baseline

arXiv:2608. 13742v1 Announce Type: cross Abstract: In LLM-based code generation, Non-Functional Requirements (NFRs) are often specified as terse one-line phrases.

By Jo\`ao Pedro Monteiro Pereira, Vinicius Cardoso Garcia

llms benchmarks

arXiv Machine Learning

Jul 22

Spaghetti Architect: A Contamination-Resistant, By-Construction-Labelled, Multi-Language Code Dataset Generator

arXiv:2607. 18642v1 Announce Type: new Abstract: Mined code corpora are abundant but uncontrolled: a snippet's semantics, surface "messiness," and difficulty are whatever the wild contained; there is no known-optimal reference to grade against; and any public sample may already sit in a model's training set.

By Yuxiang Ji

arXiv Machine Learning

Jun 2

A Grammar of Machine Learning Workflows: Rejecting Data Leakage at Call Time

arXiv:2603. 10742v4 Announce Type: replace Abstract: Data leakage has been identified in 648 published papers across 30 scientific fields.

By Simon Roth

arXiv AI

Aug 11

On the Robustness of LLMs' Internal Representation of Code Correctness

arXiv:2608. 08266v1 Announce Type: cross Abstract: Code generated by modern language models often reads naturally.

By Francisco Ribeiro, Sohaila Abdulsattar, Renata Gonzalez, Mahmoud Kassem, Sarah Nadi

llms

arXiv AI

Aug 11

Contamination Means Overestimation? A Fine-Grained Empirical Study in Code Intelligence

arXiv:2506. 02791v4 Announce Type: replace-cross Abstract: In recent years, code intelligence has gained increasing importance in the field of automated software engineering.

By Zhen Yang, Hongyi Lin, Yifan He, Junqi Wang, Zeyu Sun, Shuo Liu, Jie Xu, Pengpeng Wang, Zhongxing Yu, Qingyuan Liang

llms nlp fine-tuning

arXiv AI

Jun 11

Soft-Prompt Tuning for Fair and Efficient LLM Benchmark Evaluation

arXiv:2606. 12117v1 Announce Type: cross Abstract: Benchmark scores often misrepresent a large language model's (LLM's) knowledge, because they rely, e.

By Selen Erkan, Bastian Boll, Kristian Kersting, Bj\"orn Deiseroth, Letitia Parcalabescu

llms benchmarks

arXiv AI

Jun 29

Speculative Refinement: A Hybrid Autoregressive Diffusion Decoding Strategy and Its Behavior Across Benchmarks

arXiv:2606. 27474v1 Announce Type: cross Abstract: How should we evaluate generation systems that combine autoregressive (AR) and diffusion decoding?

By Aditi Gupta, Neel Mishra, Kushagra Trivedi, Pawan Kumar

llms diffusion benchmarks

arXiv AI

1d ago

ReLoop: Structured Modeling and Behavioral Verification for Reliable LLM-Based Optimization

arXiv:2602. 15983v3 Announce Type: replace-cross Abstract: Large language models (LLMs) can translate natural language into optimization code, but silent failures pose a critical risk: code that executes and returns solver-feasible solutions may encode semantically incorrect formulations---a feasibility--correctness gap reaching 90 percentage points on compositional problems.

By Junbo Jacob Lian, Yujun Sun, Huiling Chen, Chaoyu Zhang, Hanzhang Qin, Chung-Piaw Teo

llms fine-tuning benchmarks

arXiv AI

Jun 8

SWE-IF: Aligning Code Evaluation with Human Preference

arXiv:2510. 07315v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs) have catalyzed vibe coding, where users leverage LLMs to generate and iteratively refine code through natural language interactions until it passes their vibe check.

By Ming Zhong, Xiang Zhou, Ting-Yun Chang, Qingze Wang, Nan Xu, Xiance Si, Dan Garrette, Shyam Upadhyay, Jeremiah Liu, Jiawei Han, Benoit Schillings, Jiao Sun

llms

arXiv AI

Jun 30

Reward-Free Code Alignment from Pretrained or Fine-Tuned LLM: Unpacking the Trade-offs for Code Generation

arXiv:2606. 28998v1 Announce Type: cross Abstract: Large Language Model (LLM) alignment trains an LLM using preference data to produce outputs that better meet established quality standards.

By Gias Uddin, Sanjeepan Sivapiran

llms reinforcement-learning fine-tuning benchmarks safety

Related stories