20x Faster TRL Fine-tuning with RapidFire AI

Make LLM Fine-tuning 2x faster with Unsloth and 🤗 TRL

Accelerating PyTorch distributed fine-tuning with Intel technologies

HighTide: An Agent-Curated Open-Source VLSI Benchmark Suite

arXiv:2606. 04126v1 Announce Type: cross Abstract: We introduce HighTide, an evolving AI-assisted benchmark suite.

Parameter-Efficient Fine-Tuning using 🤗 PEFT

A Case Study of a Multi-Agent Kernel-Optimization Loop That Learns a New AI Accelerator

The article describes a case study where a large language model (LLM) agent loop was used to optimize three Gemma‑4‑12B decode kernels for the FuriosaAI RNGD NPU, a sparsely documented Rust DSL accelerator. Over 12 days on a shared device, the loop built its own target knowledge, performed microbenchmarks, recorded compiler legality rules, and iteratively refined kernel designs through role‑based search and acceptance tests. The resulting kernels achieved a 9.293× geometric mean speedup over the official baseline, each within 1.25× of the theoretical cycle minimum for weight movement.

TACO: Ternary Absolute-max Column-wise One-sparse Optimizer for LLM Fine-Tuning

The paper introduces TACO, a new optimizer for fine‑tuning large language models that drastically reduces optimizer state memory while preserving first‑order gradients. TACO selects the sign of the largest magnitude entry in each column of weight matrices, achieving a 174× reduction in persistent optimizer memory compared to AdamW8bit and a 2.9× decrease in peak training memory on OPT‑13B. This allows full‑parameter fine‑tuning of 30–32B‑parameter models on a single 80 GB GPU across multiple model families and tasks, with comparable accuracy and runtime to existing methods.

AOS: Adaptive Optimizer Switching via Training-State Signals for Faster Convergence and Better Generalization

arXiv:2608. 01997v1 Announce Type: new Abstract: Single-optimizer training is a poor fit for the distinct phases of deep network optimization: adaptive methods handle noisy early gradients well but overshoot flat minima, while SGD with momentum generalizes better in the late phase but converges slowly early on.

TideRL: Boosting Agentic RL Goodput with Readiness-Aware Scheduling

arXiv:2608. 10402v1 Announce Type: new Abstract: Reinforcement learning (RL) for large language models is moving toward multi-turn agentic workloads, where rollout tasks repeatedly pause for external environments, resume with growing contexts, and finish at highly variable times.

LLM Agent-Based Design of RISC-V CPUs with FPGA-in-the-Loop Evaluation

The paper introduces a development environment where large language model (LLM) agents design RISC‑V CPUs that are implemented on an FPGA for real‑time performance evaluation. Using on‑board measurements, the approach guides design decisions and accelerates evaluation, achieving a 9.6× speedup over single‑threaded RTL simulation. In a case study, FPGA measurements uncovered long‑run performance degradation that short diagnostic runs missed, leading to candidate selection, root‑cause analysis, and a fix.

Looping Beyond Twice: A Scalable Recipe for Looped Mixture-of-Experts

The paper introduces LOOM, a method for scaling looped mixture‑of‑experts (MoE) Transformers beyond the typical two‑loop limit. LOOM addresses two key obstacles: it stabilizes deep recurrence by bounding residual variance and re‑injecting the input embedding, and it prevents expert selection collapse by using per‑loop routers and a looping residual to maintain computational diversity. Experiments on 100 M–1.7 B parameter models show stable scaling to 9–12 loops, with significant perplexity reductions and zero‑shot accuracy gains under near‑iso‑FLOP conditions.

FlyWheel: Attention Kernels Built for the TPU

FlyWheel introduces optimized TPU kernels for softmax and linear attention, reducing vector work and improving matrix operand reuse. By overlapping independent computations, FlyWheel achieves up to 4.14× speedups over the RPA kernel and 1.80× over Splash Attention, reaching 81% of peak BF16 throughput on TPU v6e. It also boosts Gated DeltaNet prefill by up to 3.61× and enhances vLLM serving of Qwen3.8-27B prefill throughput by up to 1.56× while preserving accuracy.

A3C3: AI Algorithm and Accelerator Co-design, Co-search, and Co-generation

arXiv:2606. 20869v2 Announce Type: replace-cross Abstract: We present a holistic methodology for artificial intelligence algorithm and accelerator co-design, co-search, and co-generation (A3C3), which jointly optimizes neural network architectures and their hardware implementations to address the inefficiencies of traditional top-down AI system design flows.