Model releases, architecture work and prompting research on large language models — from frontier-lab announcements to the arXiv papers behind them.
arXiv:2609.30820v1 Announce Type: cross Abstract: Looped transformers reuse weights across recurrence steps, making low-bit quantization especially attractive. We identify two distinct failure modes...
arXiv:2609.31603v1 Announce Type: cross Abstract: Large language models (LLMs) implicitly infer attributes of their users and adapt their behavior accordingly, yet these beliefs remain difficult to i...
The paper proposes a new architecture for masked language modeling that replaces the Transformer attention mechanism with a stack of low‑rank bottleneck autoencoders. Each autoencoder mixes information locally, across the full sequence, and across attention heads, compressing and reconstructing inputs without training‑dependent width. An iterative refinement process at masked positions pulls embeddings toward a weighted neighbor average and then projects them back onto the learned manifold, achieving comparable performance to BERT with roughly 1.9× fewer FLOPs and matching BERT on rare‑token performance through a frequency‑aware training schedule.
arXiv:2609.30342v1 Announce Type: new Abstract: iKFAD is a recently proposed optimiser that replaces adaptive learning rates with adaptive friction in the momentum dynamics, yet performs as well as A...
arXiv:2609.30483v1 Announce Type: cross Abstract: Audio language models state numbers for acoustic quantities, and neither human opinion nor a judge model says whether such a number is true of the si...
arXiv:2609.30629v1 Announce Type: cross Abstract: Mission-critical UAVs increasingly rely on split vision-language model (VLM) perception under tight onboard-resource and wireless-communication const...
arXiv:2609.30738v1 Announce Type: new Abstract: KV cache eviction methods such as SnapKV and PyramidKV rank tokens solely by mean attention over a small observation window. We study a unified score,...
The paper introduces the Within-Instance Prototypical Transformer (WIPT), a method that performs single-query test-time prototype adaptation by jointly transforming an unlabelled query with labelled support embeddings to form query-specific class means. Using a frozen ViT-S/16 encoder trained on miniImageNet and evaluated on CUB, EuroSAT, and ISIC, WIPT improves 1‑shot performance on CUB and EuroSAT but not on ISIC, while in 5‑shot settings it outperforms a support‑only Transformer on ISIC but remains weaker than ProtoNet overall. The study also explores multi‑query processing, memory and latency trade‑offs, and analyzes how WIPT alters uncertain versus confident predictions, concluding that the method offers a streaming‑compatible test‑time adaptation that can enhance low‑shot cross‑domain decisions without target‑time optimization.
arXiv:2609.31122v1 Announce Type: new Abstract: Activation steering is a powerful training-free paradigm for controlling large language models at inference time. However, standard approaches estimate...
arXiv:2609.31356v1 Announce Type: new Abstract: Vision-Language Models (VLMs) exhibit remarkable zero-shot generalization, yet they often encode unwanted or hazardous stylistic domains such as ideali...
arXiv:2609.31458v1 Announce Type: new Abstract: Transformers have become a central architecture for in-context learning (ICL), particularly through their state-of-the-art performance in large languag...
arXiv:2603.19742v3 Announce Type: replace-cross Abstract: Understanding the internal mechanisms of transformer-based large language models (LLMs) is crucial for their reliable deployment and effectiv...
arXiv:2605.24981v2 Announce Type: replace Abstract: Choosing a Large Language Model (LLM) for a given task requires comparing many strong candidates, yet standard evaluation relies on costly annotati...
arXiv:2609.31234v1 Announce Type: new Abstract: Problem. Ultra-high-resolution (UHR) remote sensing with vague user intents has two bottlenecks: visual tokens are expensive, and tool calling must be...
UniPrefill is a prefill‑acceleration framework that works with virtually any model architecture, directly speeding up token‑level computation. It is implemented as a continuous‑batching operator and extends vLLM’s scheduling to support prefill‑decode co‑processing and tensor parallelism. The method delivers up to a 2.1× speedup in Time‑to‑First‑Token, with gains growing as concurrent requests increase.
The paper presents a multi‑stage framework for recognizing Kuzushiji characters in Japanese historical documents. It combines character detection, cropping, classification, reading‑order reconstruction via adaptive column clustering, and large‑language‑model‑based post‑OCR correction. The authors also augment data synthetically, correct dataset annotations, and introduce new test sets, achieving significant character error rate reductions on real, synthetic, and out‑of‑domain data.
The paper investigates the theoretical expressive power of Transformers in modeling contextual relations. By framing a text as a distribution of representations and attention as a probabilistic relation, it connects attention normalization to optimal transport: softmax yields conditional relations, while Sinkhorn yields joint relations with fixed marginals. The authors prove universal approximation results, showing that Transformers with Sinkhorn normalization can represent any joint probability relation, whereas standard softmax Transformers can represent any conditional probability relation.
The paper introduces FAST-CAP, a causality‑aware framework for simultaneous speech‑to‑speech translation that combines a factorized S2ST architecture, an adaptive policy, and a new latency metric. It employs a novel data pipeline to generate high‑fidelity, causally aligned segments, improving voice transfer and reducing the need for large training datasets. Experiments on Spanish, German, and French demonstrate that FAST‑CAP outperforms fixed‑policy baselines, achieving up to +1.2 BLEU, 26% lower latency, and a 38.8% relative latency reduction while maintaining speaker fidelity.
The paper introduces FAVoR, a method for federated personalized generation that mitigates author‑style homogenization caused by standard aggregation in parameter‑efficient fine‑tuning. Using the BlogText benchmark and ASCE diagnostics, the authors show that common federated PEFT baselines preserve semantic utility but blur author‑specific style. FAVoR employs a shared‑private adapter design, where clients upload shared updates while keeping author‑specific residual corrections locally, leading to improved style retention with minimal utility loss.
ZooWork-ShopRanker is a family of open e‑commerce rerankers (0.6B, 4B, and 8B) that align with human shopping preferences by using large language models as preference oracles to generate training pairs. The flagship 8B model serves as a teacher for the smaller 4B and 0.6B models, which are further refined on judged pairs. A new benchmark, ShopRank‑Bench, contains ~10,000 private‑traffic preference pairs and shows that all ZooWork models outperform the strongest open reranker baseline and their own un‑aligned versions.