arXiv Machine Learning October 3, 2026 By Tianshuo Qiao, Naiqian Zheng, Xiaopeng Liu, Shuguang Wang, Diandian Gu, Xuanzhe Liu, Xin Jin

FastCI: Efficient GPU-Intensive CI for LLM Training Frameworks

Read the original on arXiv Machine Learning →

FastCI is a new framework designed to make continuous integration (CI) for large language model (LLM) training frameworks more efficient. It uses runtime evidence to select and prune tests that run changed code in equivalent contexts, prioritizes high‑risk tests, and optimizes test workloads beyond each test’s intended validation scope. In evaluations on ByteDance’s LLM training framework, FastCI cut CI latency by 77.5 % and GPU usage by 63.9 %, while slightly improving modified code coverage retention.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

llms

arXiv AI

Jul 21

KernelBench-Verified: Do LLM-Generated Kernels Actually Beat PyTorch?

arXiv:2607. 16241v1 Announce Type: cross Abstract: Recent large language models (LLMs) can generate custom CUDA kernels that appear to outperform PyTorch on benchmarks such as KernelBench.

By Yunxiang Zhang (Xiangjun), Ping Yu (Xiangjun), Jianyu Wang (Xiangjun), Max (Xiangjun), Fan, Julian Reed, Azalia Mirhoseini, Will Su

llms benchmarks

arXiv AI

Aug 13

RealisticTritonBench: A Benchmark for Triton-Kernel Generation in Real-World AI Frameworks

arXiv:2608. 12004v1 Announce Type: cross Abstract: In modern AI frameworks, GPU kernels are key to overall system performance.

By Jinjun Huang, Zhongzhen Wen, Tongtong Xu, Meng Yan, Xin Xia, Zhongxin Liu

llms benchmarks

arXiv Machine Learning

Jun 5

CuTeGen: An LLM-Based Agentic Framework for Generation and Optimization of High-Performance GPU Kernels using CuTe

arXiv:2604. 01489v2 Announce Type: replace Abstract: High-performance GPU kernels are critical to modern machine learning systems, yet developing them remains a manual, expert-driven process.

By Tara Saba, Zhiyang Chen, Jikai Jason Li, Anne Ouyang, Xujie Si, Fan Long

llms agents benchmarks

arXiv AI

Aug 25

LLM4LLM: Bridging Kernel Benchmarks and Real Deployment via Closed-Loop Agentic Optimization

arXiv:2608.21836v1 Announce Type: new Abstract: Large language models have become increasingly capable agents for low-level code and kernel optimization, but isolated kernel benchmarks provide only a...

By Hui Zeng, Pengfei Yang, Yanxin Chen, Fusong Ju, Xinran Wei

llms agents benchmarks safety

arXiv AI

Jun 4

CodegenBench: Can LLMs Write Efficient Code Across Architectures?

arXiv:2606. 04023v1 Announce Type: cross Abstract: While large language models (LLMs) have been extensively evaluated on code generation tasks for general-purpose programming and GPU-accelerated environments (e.

By Jie Li, Wenzhao Wu, Junqi Hu, Qinrui Zheng, Bowen Wu, Juepeng Zheng, Yutong Lu, Haohuan Fu

llms benchmarks

Hugging Face Trending Papers

Aug 18

PTXBench: Benchmark and Adapt LLMs for GPU Kernel Optimization with Architecture-specific PTX

PTXBench is a benchmark designed to evaluate and adapt large language models (LLMs) for GPU kernel optimization using architecture-specific PTX code. It assesses functional correctness, runtime execution of target instructions, and speedup over leading libraries on GEMM and attention workloads on H100 and B200 GPUs. The study finds uneven performance across models, especially on complex attention backward tasks, and shows that fine‑tuning Qwen3.6‑27B improves some tasks but generalization remains inconsistent.

llms fine-tuning benchmarks

One email a morning, machine-written

Related stories

KernelBench-Verified: Do LLM-Generated Kernels Actually Beat PyTorch?

RealisticTritonBench: A Benchmark for Triton-Kernel Generation in Real-World AI Frameworks

CuTeGen: An LLM-Based Agentic Framework for Generation and Optimization of High-Performance GPU Kernels using CuTe

LLM4LLM: Bridging Kernel Benchmarks and Real Deployment via Closed-Loop Agentic Optimization

CodegenBench: Can LLMs Write Efficient Code Across Architectures?

PTXBench: Benchmark and Adapt LLMs for GPU Kernel Optimization with Architecture-specific PTX