arXiv Machine Learning June 30, 2026 By John Sweeney

Optimizer Memory Makes Shuffle Order a First-Order Source of Fine-Tuning Noise

Read the original on arXiv Machine Learning →

arXiv:2606. 29554v1 Announce Type: new Abstract: Shuffle order can be a larger source of fine-tuning noise than a memoryless analysis predicts: fixed-clock optimizer memory makes local equal-multiset contrasts first order in the learning rate rather than second order, and the resulting order channel can be large enough for a single seed to flip a close A/B comparison.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

fine-tuning

arXiv Machine Learning

Aug 26

AQLoRA: A Zero-Search Recipe for Fast Quantized LoRA Fine-Tuning

arXiv:2608.23816v1 Announce Type: new Abstract: Quantized fine-tuning (QLoRA) saves memory but not time. It dequantizes every 4-bit weight on the fly, so it trains more slowly than fp16 LoRA. We pres...

By Md Romyull Islam

fine-tuning efficiency

arXiv Machine Learning

Jul 7

Variable Bit-width Quantization: Learning Per-Group Precision for "Bigger-but-Smaller" Language Models

arXiv:2607. 02893v1 Announce Type: new Abstract: Low-bit quantization shrinks language models but treats precision as a single global hyper-parameter: every weight uses the same bit-width.

By Hamish Ogilvy

llms efficiency

arXiv Machine Learning

Aug 26

StateTune: Transforming LLM-Assisted EDA Flow Tuning into a Stateful, Closed-Loop Process

arXiv:2608.23601v1 Announce Type: cross Abstract: EDA flow parameter tuning is critical for quality-of-results~(QoR), yet the parameter space is large, tightly coupled, and full evaluations are prohi...

By Kunlong Li, Shangshang Yao, Su Zheng, Lingli Wang

llms rag benchmarks

arXiv Machine Learning

Aug 4

On the Limits of Machine-Learned Ranking for Modern Microarchitectural Policies

arXiv:2608. 01041v1 Announce Type: cross Abstract: Machine-learning predictors estimate processor performance far faster than cycle-level simulation.

By Yanxin Zhang, Shayne Wadle, Yuxuan Xiong, Zheyu Fu, Trivikram Krishnamurthy, Karu Sankaralingam

arXiv AI

Oct 15

StructZO: Structured Zeroth-Order Optimization for Memory-Efficient LLM Fine-Tuning

arXiv:2610.16451v1 Announce Type: cross Abstract: Fine-tuning large language models at scale is increasingly limited by the memory cost of backpropagation. Zeroth-Order (ZO) optimizers reduce this bu...

By Xuanlin Yang, Jia Li, Zhouchen Lin

llms fine-tuning

arXiv Machine Learning

Jul 21

PsiLogic: Chaos-Aware Active Cancellation for Adam with a Fair Cross-Domain Benchmark

arXiv:2607. 16268v1 Announce Type: new Abstract: Adaptive optimizers such as Adam and AdamW apply the same update rule regardless of whether training is in a chaotic early phase or near convergence.

By Ali Sultonov

llms diffusion nlp benchmarks

One email a morning, machine-written

Related stories

AQLoRA: A Zero-Search Recipe for Fast Quantized LoRA Fine-Tuning

Variable Bit-width Quantization: Learning Per-Group Precision for "Bigger-but-Smaller" Language Models

StateTune: Transforming LLM-Assisted EDA Flow Tuning into a Stateful, Closed-Loop Process

On the Limits of Machine-Learned Ranking for Modern Microarchitectural Policies

StructZO: Structured Zeroth-Order Optimization for Memory-Efficient LLM Fine-Tuning

PsiLogic: Chaos-Aware Active Cancellation for Adam with a Fair Cross-Domain Benchmark