Model releases, architecture work and prompting research on large language models — from frontier-lab announcements to the arXiv papers behind them.
arXiv:2609.39229v1 Announce Type: cross Abstract: Automatic evaluation of faithfulness increasingly relies on a large language model acting as a judge, yet the most reliable judges are proprietary fr...
The paper introduces Conditional Functional Substitutability (CFS) as a new way to measure redundancy in Transformers by examining when intermediate states produce similar downstream responses. CFS uncovers functional relationships and potential reductions that traditional importance- or similarity-based metrics miss, revealing systematic reorganization as models scale. Experiments across modalities and Transformer families show that performance gains do not always align with increased substitutability, and that models with more independent functional structure perform better, offering a functional explanation for diminishing returns and enabling more efficient dynamic computation.
The paper investigates transferring specialized task-oriented behavior to a general language model without training or distillation. It applies two training‑free heterogeneous merging techniques—Intersection‑Merge (IM) and Activate‑Prune‑Merge (APM)—to project a specialist donor into the recipient’s parameter space and interpolate backbone weights. Experiments across embedding, reranking, reward modeling, and MoE code‑specialist tasks show that both methods improve the general model, demonstrating that simple parameter‑level merging can transfer capabilities across diverse specialist roles.
arXiv:2609.39588v1 Announce Type: new Abstract: We push the frontier of large-scale spatial intelligence in Vision-Language Models (VLMs) and introduce the first benchmark that probes geographical la...
GroundAnything is a 4‑B parameter grounding foundation model that combines autoregressive and diffusion approaches to achieve fast parallel decoding while maintaining precise visual grounding. By treating grounding as visual evidence extraction and using blockwise denoising, it allows spatial hypotheses to be generated in parallel and refined iteratively. The model outperforms existing state‑of‑the‑art methods on 30 grounding benchmarks, achieving 72.42% accuracy with its autoregressive variant and 61.75% with entropy‑guided decoding, while also offering significant speedups through optional self‑speculative decoding.
arXiv:2609.39645v1 Announce Type: cross Abstract: Multi-agent collaboration lets large language models (LLMs) improve question answering through deliberation and feedback. Yet shared discussion coupl...
arXiv:2609.39701v1 Announce Type: new Abstract: Value steering should change an LLM's normative priorities while preserving the scenario, facts, and task constraints underlying its answer. Convention...
arXiv:2609.39836v1 Announce Type: new Abstract: Contrastive vision-language models map visual and textual representations into a shared normalized embedding space, making cosine similarity the natura...
GrammarRL introduces a label‑free reinforcement learning approach that adapts language models to grammar constraints without annotated data. It optimizes two self‑supervised rewards—direct and reverse—using a Reinforce Leave‑One‑Out objective over grammar‑constrained rollouts, and regularizes toward a frozen base model. Experiments on sign‑language gloss translation, hierarchical text classification, and named entity recognition with Llama models show consistent gains over constrained greedy decoding and competitive performance to beam search while keeping inference cost low.
arXiv:2609.39884v1 Announce Type: cross Abstract: Role prompting elicits specialized behavior from large language models through an expert identity, offering a lightweight way to guide reasoning on d...
The paper introduces the Cluster Attention Neural Operator (CANO), a neural operator that uses a novel cross‑attention mechanism to dynamically cluster queries while keeping full‑resolution keys and values. This design eliminates the slice compression and weight‑sharing limitations of previous Transformer‑based operators, maintaining fast computation and global interactions. Experiments on a range of fluid and solid dynamics benchmarks—including Navier‑Stokes, Airfoil, Plasticity, Pipe Turbulence, and Composites—show that CANO achieves lower errors than existing baselines and demonstrates strong geometric adaptability and temporal consistency.
The study evaluates whether a fine‑tuned open‑weight model (Gemma‑3‑12B) can match the performance of GPT‑4o in extracting multi‑label intracranial hemorrhage acuity from non‑contrast head‑CT reports. Using a 2×2 design that varied adaptation strategy (classification head vs. instruction fine‑tuning) and training‑data source (distilled real GPT‑4o labels vs. synthetic GPT‑4o‑generated reports), the distilled instruction‑tuned model achieved macro‑F1 scores comparable to GPT‑4o and surpassed the untuned base model. The key finding is that the source of training data—distilled real reports—was more important than the fine‑tuning method, and that the entire fine‑tuning and inference process fits on a single 24 GB consumer GPU.
arXiv:2609.40305v1 Announce Type: new Abstract: Improving text-to-image models has traditionally relied on increasing model size or the number of denoising steps. In this work, we explore an alternat...
The paper explores ways to reduce the memorization of training data in language models, testing three regularizer-based, three finetuning-based, and eleven machine unlearning methods—five of which are newly introduced. It introduces TinyMem, a lightweight suite of small models for rapid testing of these mitigation techniques, and shows that unlearning methods, particularly BalancedSubnet, outperform others in removing memorized content while maintaining task performance. The study also finds that regularizer-based approaches are slow and ineffective, while finetuning methods are costly, especially when high accuracy is required.
The paper introduces EDGC, an entropy-driven dynamic gradient compression framework designed to reduce communication overhead during large language model training. EDGC adapts compression ranks based on gradient entropy, using efficient entropy estimation, a theoretical entropy-to-rank model, and window-based rank adjustments across pipeline stages. Experiments on GPT‑2 models with 2.5B and 12.1B parameters on 32‑V100 and 64‑H100 GPU clusters demonstrate up to 46.45% lower communication latency and a 16.13% reduction in end‑to‑end training time while preserving model quality.
The paper presents efficient approximations for key statistics of the Neural Tangent Kernel (NTK) in finite-width neural networks using randomized trace estimation (Hutch++). It demonstrates that the NTK trace, Frobenius norm, effective rank, and alignment can be estimated with high accuracy via matrix-free products, leveraging the NTK’s positive-semidefinite structure to use one-sided estimators with forward or reverse-mode differentiation. Experiments on MLPs, GRUs, and a 410‑million‑parameter Transformer show orders‑of‑magnitude speedups and enable practical state‑space NTK diagnostics at large scales, including applications to RNN training and data‑scarce knowledge distillation.
arXiv:2604.16778v3 Announce Type: replace-cross Abstract: Modern agents specialize in varying domains while there is no clear approach combining different domain skills. We propose a federated learni...
arXiv:2607.14576v2 Announce Type: replace-cross Abstract: Whether a deep residual architecture trains stably is usually determined by training it, which is expensive and answers the question only for...
arXiv:2506.21734v4 Announce Type: replace Abstract: Reasoning, the process of devising and executing complex goal-oriented action sequences, remains a critical challenge in AI. Current large language...
arXiv:2511.00421v2 Announce Type: replace-cross Abstract: Large language models (LLMs) show promise in medical applications, but their ability to detect and correct errors in clinical texts remains u...