The paper develops a statistical theory for minimum‑norm interpolation in high‑dimensional regression, showing how regularization geometry and signal sparsity affect generalization. It identifies regimes where sparsity‑promoting regularizers yield exact interpolation that is far more accurate than approximate fitting, and proves a zero–one generalization law for strongly overparameterized noiseless problems. The authors also characterize training and generalization errors along ρ‑regularization paths when feature dimension and sample size are proportional, demonstrating that generalization improves with more sparsity‑promoting norms and sparser targets, and that small changes in regularization strength can cause large shifts in generalization.
whyItMatters":"The work provides a quantitative understanding of delayed generalization (grokking) and reveals a statistical instability in minimum‑norm interpolation, offering insights that could guide the design of regularizers for better generalization in overparameterized models."
By Gil Kur, Ileana Rugina, Cl\'ementine Carla Juliette Domin\'e, Marco Mondelli
arXiv:2609.38523v1 Announce Type: cross
Abstract: Financial forecasting from earnings conference calls requires models to reason over complex corporate disclosures, market expectations, and subtle co...
By Dong Shu, Yanguang Liu, Huopu Zhang, Saisai Hu, Haiyan Zhao, Hekun Huang, Mengnan Du
arXiv:2609.38536v1 Announce Type: cross
Abstract: Diffusion-based large language models (dLLMs) promise to break the sequential latency bottleneck of autoregressive agents through parallel decoding,...
By Jiacheng Qiu, Christopher E. Mower, Jan Peters, Haitham Bou-Ammar, Matthieu Zimmer
arXiv:2609.38672v1 Announce Type: cross
Abstract: Beam-search-based test-time methods provide an effective way to improve large language model (LLM) performance on long-horizon generation by pruning...
By Qijia He, Yu Huang, Yuan Cheng, Yuxin Chen, Yingbin Liang
ChartDensity-Bench is a benchmark designed to evaluate multimodal large language models (MLLMs) on their ability to reconstruct structured numerical data from scientific charts that vary in visual density. The benchmark uses charts paired with source-level ground-truth data and systematically changes the number of simultaneously presented charts (k = 1, 3, 6, 9) to assess how density affects reconstruction performance. A multi‑dimensional evaluation framework measures structural reliability, reconstruction completeness, parseability, and numerical fidelity, revealing that numerical reconstruction generally worsens as visual density increases, with varying degrees of degradation across different models.
By Xinhe Wu, Yadong Jin
arXiv:2609.38860v1 Announce Type: cross
Abstract: Learning from human preferences is central to large language model (LLM) alignment, but human preference annotation is costly. Active preference lear...
By Zhongman Du, Huiming Zhang, Haodong Zhu, Baochang Zhang
arXiv:2609.38879v1 Announce Type: cross
Abstract: Large language models rely heavily on human text, which often conveys surface answers rather than the spatial and structural logic behind them. Prote...
By Yong Liu, Zhanpeng Shi, Yizhou Dang, Zhongyue Zhang, Xiaoliang Shi, Zhijian Wei, Shuangjia Zheng
The paper introduces OptTips, a knowledge base of 50 expert optimization modeling techniques, and OptDachshund, a multi‑agent framework that generates mathematical models and solver code from natural‑language problem descriptions. Using these tools, the authors create the EfficientOpt benchmark, comprising 561 expert‑reviewed tasks with paired reference implementations, to evaluate large language models (LLMs) on both correctness and computational efficiency. Their experiments with 11 LLMs show a consistent efficiency gap: even when LLMs produce correct solutions, the resulting programs often take longer to solve than expert‑crafted counterparts, highlighting the need to assess both accuracy and runtime performance in LLM‑based optimization modeling.
By Zhong Li, Xin Huang, Jinhui Wan, Xiangyi Wang, Shenkai Zhang, Ruiqi Chen, Wenyu Liu, Zaiwen Wen, Ziyan Luo
The paper introduces Unmerge, an efficient machine unlearning algorithm that treats unlearning as the inverse of task arithmetic. By representing the forget component as a low‑rank basis at each layer, Unmerge optimizes three goals—matching the merged vector, suppressing leakage, and bounding correction size—to limit forget leakage and retain damage. Experiments on ResNet‑50, ViT‑S/16, and Llama‑3.2‑3B show significant performance gains over existing methods while maintaining privacy and feature‑distribution fidelity.
By Haoran Tang, Andrew Tan, Rajiv Khanna
The paper introduces PACT, a method for unlearning deceptive behaviors in large language models by using contrastive forget sets that compare a model’s responses under deceptive and neutral contexts. PACT trains the model to produce pressure‑aware counterfactual targets, preserving benign system‑prompt adherence and reasoning traces while dramatically reducing deception rates from over 50% to under 3% on 32B reasoning models.
By Haoran Tang, Rajiv Khanna
arXiv:2609.39321v1 Announce Type: cross
Abstract: Group Relative Policy Optimization (GRPO) has emerged as a memory-efficient reinforcement fine-tuning (RFT) technique for reasoning-intensive tasks....
By Rajat Ghosh, Vaishnavi Bhargava, Henry Wong, Aryan Singhal, Debojyoti Dutta
The paper introduces GFD-OPD, a method for on‑policy distillation of diffusion models that addresses challenges when compressing large teachers into smaller students. It identifies that standard distillation fails due to distribution gaps and classifier‑free guidance amplification, and proposes Fixed‑State KL to measure these gaps. GFD‑OPD reduces the student‑teacher discrepancy and achieves state‑of‑the‑art performance across multiple benchmarks.
By Zhenxing Zhang, Jiayan Teng, Wenxu Wu, Zhuoyi Yang, Jiazheng Xu, Wendi Zheng, Jie Tang, Dan Guo, Meng Wang
The paper proposes an inference auction for large language model (LLM) APIs, enabling users to bid for faster service when compute demand exceeds capacity. The auction allocates priority efficiently without increasing latency, and includes fast algorithms for truthful bidding and an autobidding agent that adjusts bids within a user’s budget to maximize utility. Experiments show the auction improves system welfare while preserving the cache utilization and latency benefits of the SGLang inference framework.
By Keegan Harris, Siddharth Prasad, Asher Trockman, Nika Haghtalab, Michael I. Jordan
arXiv:2609.40235v1 Announce Type: cross
Abstract: Continuous diffusion language models generate all tokens in parallel, yet high-quality generation can still require hundreds of network evaluations (...
By Paul Le Van Kiem, Dario Shariatian, Umut Simsekli, Alain Durmus
arXiv:2609.40316v1 Announce Type: cross
Abstract: Looped transformers and Mixture-of-Experts (MoE) offer complementary routes to efficient scaling: recurrence increases computational depth at fixed p...
By Yanbei Chen, Anirudh Goyal, Raghuraman Krishnamoorthi
arXiv:2609.40361v1 Announce Type: cross
Abstract: Multimodal large language models (MLLMs) are rapidly advancing clinical diagnosis, yet their adaptation pipelines remain anchored to accuracy-based o...
By Tian Xia, Minghao Liu, Yiqing Liang, Laixi Shi, Jiayun Wang
arXiv:2609.38274v1 Announce Type: new
Abstract: The performance ceiling of an LLM team is constrained not only by individual model capabilities, but also by inter-member error resonance and predictiv...
By Liangyu Teng, Hengsong Liu, Juncen Guo, Jingyu Zhang, Yang Liu, Jing Liu, Liang Song
arXiv:2609.38364v1 Announce Type: cross
Abstract: Discrete diffusion models and flow matching have emerged as powerful frameworks for generative modeling over discrete state spaces, yet efficient few...
By Yidong Ouyang, Zhengyan Wan, Themis Haris, Tian Tan, Liqian Peng, Henry Li, Ziqian Lin, Jianhang Chen, Maryam Karimzadehgan, Alec Go, George Michailidis
Doc2LoRA introduces a method to represent scientific papers as LoRA adapters generated by a hypernetwork, enabling each point in the resulting vector space—whether a real paper or a mixture—to be interpreted by a large language model (LLM). The approach allows both search and generation: it achieves competitive search performance with models like SPECTER2 and EmbeddingGemma, while also enabling the LLM to label subfields and generate abstracts that smoothly transition between papers. The invertible transform ensures every point maps back to an LLM, making the embeddings usable for both querying and creative idea generation.
By Chand Sahil Mansuri, Joel Zachariah, Sadamori Kojaku
arXiv:2609.38477v1 Announce Type: cross
Abstract: Large language models (LLMs) incur substantial storage, memory-bandwidth and energy costs, motivating compact weight representations. Existing seed-b...
By Qiuyu Ren, Sudipta Paria, Aritra Dasgupta, Swarup Bhunia