arXiv AI By Kaijian Zou, Aaron Xiong, Yunxiang Zhang, Frederick Zhang, Yueqi Ren, Jirong Yang, Ayoung Lee, Shitanshu Bhushan, Lu Wang

LiveOIBench: Can Large Language Models Outperform Human Contestants in Informatics Olympiads?

Read the original on arXiv AI →

arXiv:2510. 09595v3 Announce Type: replace Abstract: Competitive programming problems are increasingly used to evaluate the coding capabilities of large language models (LLMs) due to their complexity and ease of verification.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI
Sep 3

Post-Training Language Models for Gold-Medal Performance in Coding Competitions

The paper reports a specialized training pipeline for large language models to excel in competitive programming, combining problem curation, synthetic reasoning traces, supervised fine‑tuning, and reinforcement learning. Using 22,000 curated problems, the authors train two models—Nemotron‑3‑Nano‑CC (30B) and Nemotron‑3‑Ultra‑CC (550B)—and introduce GenCorrect, a test‑time refinement strategy. On the IOI 2025 benchmark, Nano‑CC scores 468 points with GenCorrect, surpassing the gold‑medal threshold, while Ultra‑CC reaches 502; in IOI 2026, a competition‑specific Ultra‑CC system scores 535.4, exceeding both the gold threshold and the top human score of 498.27, marking the first AI system to outscore the highest‑scoring human contestant on an IOI problem set.

By Aleksander Ficek, Sean Narenthiran, Mehrzad Samadi, Somshubra Majumdar, Boris Ginsburg
arXiv Computation and Language
Sep 1

SciOrch: Learning to Orchestrate Expert LLMs for Solving Frontier Multimodal Scientific Reasoning Tasks

arXiv:2606.15872v2 Announce Type: replace Abstract: Frontier scientific reasoning remains a major challenge for large language models (LLMs), where even the strongest commercial systems fall short of...

By Jingru Guo, Xiangyuan Xue, Lian Zhang, Wanghan Xu, Siki Chen, Philip Torr, Wanli Ouyang, Lei Bai, Zhenfei Yin
arXiv AI
Jun 30

SWE-fficiency: Can Language Models Optimize Real-World Repositories on Real Workloads?

arXiv:2511. 06090v3 Announce Type: replace-cross Abstract: Optimizing the performance of large-scale software repositories demands expertise in code reasoning and software engineering (SWE) to reduce runtime while preserving program correctness.

By Jeffrey Jian Ma, Milad Hashemi, Amir Yazdanbakhsh, Kevin Swersky, Ofir Press, Enhui Li, Vijay Janapa Reddi, Parthasarathy Ranganathan
arXiv AI
2d ago

Benchmarking Prompt Optimization of Large Language Models With Chess

arXiv:2610. 00416v1 Announce Type: new Abstract: Evaluating large language models becomes increasingly challenging as their capabilities advance: benchmarks can saturate, public test sets risk contamination, and assessing harder tasks can require expensive grading or execution infrastructure.

By Timoth\'ee Lesort, Alejandra L\'opez de Aberasturi G\'omez, Tristan Karch, Tom Veniat, Philippe Modard, Karl Tuyls, Ludovic Denoyer
arXiv AI
Sep 25

LiveMathematicianBench: A Live Benchmark for Research-Level Mathematical Reasoning with Proof Sketches

LiveMathematicianBench is a dynamic multiple‑choice benchmark for research‑level mathematical reasoning, built from recent arXiv papers published after model training cutoffs. It introduces a thirteen‑category logical taxonomy of theorem types and uses a proof‑sketch‑guided distractor pipeline to create plausible but invalid answer choices, enhancing sensitivity to genuine reasoning. Evaluation shows current large language models perform poorly, with the best model scoring 43.5% overall and only 17.6% under substitution‑resistant conditions, indicating the benchmark’s difficulty and realism.

By Linyang He, Qiyao Yu, Hanze Dong, Baohao Liao, Xinxing Xu, Micah Goldblum, Jiang Bian, Nima Mesgarani