Model releases, architecture work and prompting research on large language models — from frontier-lab announcements to the arXiv papers behind them.
arXiv:2609.38860v1 Announce Type: cross Abstract: Learning from human preferences is central to large language model (LLM) alignment, but human preference annotation is costly. Active preference lear...
arXiv:2609.38879v1 Announce Type: cross Abstract: Large language models rely heavily on human text, which often conveys surface answers rather than the spatial and structural logic behind them. Prote...
The paper introduces OptTips, a knowledge base of 50 expert optimization modeling techniques, and OptDachshund, a multi‑agent framework that generates mathematical models and solver code from natural‑language problem descriptions. Using these tools, the authors create the EfficientOpt benchmark, comprising 561 expert‑reviewed tasks with paired reference implementations, to evaluate large language models (LLMs) on both correctness and computational efficiency. Their experiments with 11 LLMs show a consistent efficiency gap: even when LLMs produce correct solutions, the resulting programs often take longer to solve than expert‑crafted counterparts, highlighting the need to assess both accuracy and runtime performance in LLM‑based optimization modeling.
The paper introduces Unmerge, an efficient machine unlearning algorithm that treats unlearning as the inverse of task arithmetic. By representing the forget component as a low‑rank basis at each layer, Unmerge optimizes three goals—matching the merged vector, suppressing leakage, and bounding correction size—to limit forget leakage and retain damage. Experiments on ResNet‑50, ViT‑S/16, and Llama‑3.2‑3B show significant performance gains over existing methods while maintaining privacy and feature‑distribution fidelity.
The paper introduces PACT, a method for unlearning deceptive behaviors in large language models by using contrastive forget sets that compare a model’s responses under deceptive and neutral contexts. PACT trains the model to produce pressure‑aware counterfactual targets, preserving benign system‑prompt adherence and reasoning traces while dramatically reducing deception rates from over 50% to under 3% on 32B reasoning models.
arXiv:2609.39321v1 Announce Type: cross Abstract: Group Relative Policy Optimization (GRPO) has emerged as a memory-efficient reinforcement fine-tuning (RFT) technique for reasoning-intensive tasks....
The paper introduces GFD-OPD, a method for on‑policy distillation of diffusion models that addresses challenges when compressing large teachers into smaller students. It identifies that standard distillation fails due to distribution gaps and classifier‑free guidance amplification, and proposes Fixed‑State KL to measure these gaps. GFD‑OPD reduces the student‑teacher discrepancy and achieves state‑of‑the‑art performance across multiple benchmarks.
The paper proposes an inference auction for large language model (LLM) APIs, enabling users to bid for faster service when compute demand exceeds capacity. The auction allocates priority efficiently without increasing latency, and includes fast algorithms for truthful bidding and an autobidding agent that adjusts bids within a user’s budget to maximize utility. Experiments show the auction improves system welfare while preserving the cache utilization and latency benefits of the SGLang inference framework.
arXiv:2609.40235v1 Announce Type: cross Abstract: Continuous diffusion language models generate all tokens in parallel, yet high-quality generation can still require hundreds of network evaluations (...
arXiv:2609.40316v1 Announce Type: cross Abstract: Looped transformers and Mixture-of-Experts (MoE) offer complementary routes to efficient scaling: recurrence increases computational depth at fixed p...
arXiv:2609.40361v1 Announce Type: cross Abstract: Multimodal large language models (MLLMs) are rapidly advancing clinical diagnosis, yet their adaptation pipelines remain anchored to accuracy-based o...
arXiv:2609.38274v1 Announce Type: new Abstract: The performance ceiling of an LLM team is constrained not only by individual model capabilities, but also by inter-member error resonance and predictiv...
arXiv:2609.38364v1 Announce Type: cross Abstract: Discrete diffusion models and flow matching have emerged as powerful frameworks for generative modeling over discrete state spaces, yet efficient few...
Doc2LoRA introduces a method to represent scientific papers as LoRA adapters generated by a hypernetwork, enabling each point in the resulting vector space—whether a real paper or a mixture—to be interpreted by a large language model (LLM). The approach allows both search and generation: it achieves competitive search performance with models like SPECTER2 and EmbeddingGemma, while also enabling the LLM to label subfields and generate abstracts that smoothly transition between papers. The invertible transform ensures every point maps back to an LLM, making the embeddings usable for both querying and creative idea generation.
arXiv:2609.38477v1 Announce Type: cross Abstract: Large language models (LLMs) incur substantial storage, memory-bandwidth and energy costs, motivating compact weight representations. Existing seed-b...
arXiv:2609.38530v1 Announce Type: new Abstract: Language models increasingly use architectures that vary attention span and positional encoding across layers, such as applying RoPE with sliding-windo...
arXiv:2609.38543v1 Announce Type: new Abstract: Constantly evolving real-world knowledge necessitates models to be updated continuously. Especially in medicine, as clinical evidence changes over time...
arXiv:2609.38604v1 Announce Type: cross Abstract: Modern LLM agents increasingly tackle complex tasks through interactive, long-horizon exchanges with users, while existing benchmarks generally assum...
arXiv:2609.38792v1 Announce Type: new Abstract: We study training LLM judges from natural language feedback, especially for subjective tasks where the verdict depends strongly on which evaluation cri...
arXiv:2609.38982v1 Announce Type: cross Abstract: Coding agents powered by large language models (LLMs) have shown remarkable abilities to autonomously reason about and achieve goals in the digital w...