The paper proves that deep residual self‑attention networks can universally interpolate between any two collections of sequences using only two fixed single‑head attention blocks with Gaussian‑initialized projections. The interpolation is achieved by varying the order, signs, and durations of these blocks, independent of the specific input and output sequences. The result holds for both continuous and finite depth, and the authors also extend the analysis to causal‑masked settings.
By Sibylle Marcotte, Joan Bruna
Faynt is a family of Transformer policies (10M and 75M parameters) that control all 26 characters in Super Smash Bros. Melee from a single checkpoint. After reinforcement learning, the 10M model wins 98.4% of same‑character games against fourteen specialist and multi‑character releases, and defeats a zero‑delay Slippi‑AI model in all 68 evaluated games. The work details architecture, scaling, hyperparameter transfer, supervised pretraining on 840,000 human replays, post‑training curricula, distillation, and efficient inference, and it releases the weights, benchmark suites, and a platform for automated model tournaments.
By Ali Janati, Nikita Kuzmin, Rohit Swamy, Charles Niu
The paper introduces ZFO, a lightweight framework that separates direction selection from step-size determination in large‑scale neural network optimization. ZFO uses a trusted first‑order optimizer to pick a search direction and then performs only two additional objective evaluations to build a local curvature‑aware model, selecting an adaptive step within a bounded interval. The authors provide theoretical guarantees for reliable curvature estimation, near‑optimal step selection, and convergence to a stationary point, and demonstrate that ZFO improves optimization and final performance over fixed‑step first‑order baselines on language‑model fine‑tuning tasks.
By Cristian McGee, El Houcine Bergou, Aritra Dutta
CAST (Cost‑Aware Speculative Trees) is a method that improves speculative decoding for large language models by packing multiple drafted token candidates into a tree and verifying the entire tree in a single target‑model pass, rather than only the top‑scoring chain. The tree width is adaptively chosen based on a latency measurement, ensuring that each added candidate’s expected gain outweighs its verification cost. Experiments across five domains, three GPU generations, and two model families show that CAST can be up to 43 % faster than the standard chain, while preserving the target model’s output distribution under both greedy and sampled decoding.
By Jungseob Lee, Sugyeong Eo
The paper introduces an unsupervised algorithm that separates content and style in language model outputs, enabling explicit style control. Applied to over 100K traces from nine teacher models, the method uncovers six recurring, imbalanced styles and fine‑tunes student models to follow them. This style conditioning improves Pass@$k$ on six math reasoning benchmarks and shows that style influences problem‑solving success, offering a new avenue for controlling and enhancing model performance.
By Ioana Marinescu, Eric Karl Oermann, Kyunghyun Cho
StabilityArc is a method that decodes protein sequence embeddings into generalizable stability landscapes. It uses a shared RoPE transformer to map frozen ESMC-600M residue representations into an Lx20 matrix of substitution effects, with a symmetric, contact-aware residual to predict epistasis. In extensive leave-one-protein-out tests on 134,794 ProteinGym variants, StabilityArc achieves a Spearman correlation of 0.7134, surpassing the best zero‑shot baseline, and further improves Kermut’s performance when used as a prior.
By Aaron L. Feller, Andrew D. Ellington, Claus O. Wilke
VERITYGATE is a four‑gate framework that checks whether LLM‑generated narrations adhere to a fixed schema of declared evidence IDs, entities, numbers, and claim types, rather than verifying every fact in the prose. In experiments with GPT‑4o‑mini, Llama‑3.3‑70B, and Claude Sonnet 4.6 on 900 grounded‑ungrounded pairs, the framework identified high failure rates (up to 80.3% for mini claims) and demonstrated that a single repair pass can improve claim survival rates. The authors also provide code, data, and preliminary human studies to validate the rules and highlight gaps between schema compliance and correct prose.
"whyItMatters":"The framework offers a systematic way to evaluate and improve the faithfulness of LLM explanations to structured evidence, revealing significant failure rates and the impact of repair strategies."
By Sachin Gupta
The paper investigates how sentence‑specificity scores can guide the selection of revisions in collaborative technical documentation. It compares two predictors—SpeciTeller and a target‑adapted model by Ko et al.—across Wikipedia and three technical corpora, finding that the predictors rank sentences differently and that SpeciTeller can improve direction‑valid selection rates in certain datasets. The study also shows that filtering and token‑length adjustments alter but do not reconcile these ranking differences.
By Rocker D'Antonio, Thomas Benton Townsend, Dimitrios Michael Manias
TRACE tackles real‑world dynamic resource assignment by combining evolutionary automatic heuristic design with an agentic knowledge‑extraction workflow. A Reasoner agent interprets system logs to hypothesize about underlying dynamics, while a Coder agent generates and runs schema‑specific code to validate these hypotheses, producing insights or executable tools for the evolved heuristics. Evaluations on a synthetic cloud benchmark and a 5G vRAN scenario show that TRACE outperforms existing AHD methods, delivering more auditable heuristics with less than 2% overhead.
By Jose A. Ayala-Romero, Andres Garcia-Saavedra, Xavier Costa-Perez
The paper introduces LAST, a Looped Audio Spectrogram Transformer that processes all tokens once and then reuses the same blocks to refine only the class token over fixed audio features, making subsequent passes inexpensive. On AudioSet, a ten‑pass LAST outperforms a twelve‑layer sequential transformer by 2.1% relative mean average precision while using 49.4% fewer parameters, 42% fewer MACs, and achieving 9.8% higher throughput. Increasing the pass count from two to ten improves accuracy with only a 1.2% increase in computation, and the model shows enhanced robustness to temporal masking and other auditory augmentations across music, environmental, and event sound classification tasks.
By Haider Al-Tahan, Sean O'Brien, Anastasia Razdaibiedina, N. Apurva Ratan Murty
ConQuR introduces a lightweight post‑training rotation calibration for large language model activation quantization. By learning orthogonal rotations that align normalized activations with the corners of an inscribed hypercube, the method distributes activation energy evenly and can be updated online without storing activations. Experiments on Llama‑2 and Llama‑3 models (3B–70B) show competitive or improved perplexity and reasoning performance while avoiding costly training or large offline storage.
By Chayne Thrash, Ali Abbasi, Soheil Kolouri
The paper proposes constraining the query and key projection matrices in Transformer attention to the Stiefel manifold and optimizing them with a Riemannian Adam optimizer. It demonstrates that this geometric constraint yields significant performance gains on a CIFAR‑10 patch benchmark, with the constrained model outperforming standard AdamW by up to +6.79 percentage points. The authors also show that weight decay has no effect on the constrained frames and that the improvement is driven by a scale‑free step size rather than the manifold projection or equivariance properties.
By Rub\'en Dar\'io Guerrero
The paper proposes a sensitivity‑aware residual‑stream pruning method for large language models that goes beyond minimizing activation reconstruction error. By using a second‑order approximation of output KL divergence, the authors derive a spectral upper bound that selects pruning subspaces based on both activation covariance and output sensitivity, enabling efficient eigendecomposition. Experiments on instruction‑tuned language models show that this approach consistently reduces calibration KL divergence, improves perplexity, and enhances downstream task performance across various compression levels.
By Chayne Thrash, Kevin Chen, Soheil Kolouri
The paper introduces MERITED, a framework that combines Instance-Based Learning Theory (IBLT) with Energy Based Models (EBMs) to enable metacognitive reasoning about computational effort. It allows an EBM to dynamically allocate resources based on uncertainty estimates, addressing limitations of large language models that cannot predict uncertainty before responding. The authors present a 191‑million‑parameter reasoning EBM and demonstrate how MERITED uses an IBL model for efficient, uncertainty‑driven compute allocation.
By Tailia Malloy, Prateek Kumar Rajput, Serge Lionel Nikiema, Cleotilde Gonzalez, Tegawend\'e F. Bissyand\'e
The paper introduces the Denoising Workload Surface (DWS), a two‑dimensional probability surface that captures the block‑autoregressive generation structure of diffusion large language models (dLLMs). By preserving both output block and within‑block denoising step information, DWS enables a lightweight, prompt‑only predictor to estimate per‑request inference cost accurately, even on a single CPU core. In real‑world serving experiments, DWS reduces cost‑prediction error by up to 2.5× and improves end‑to‑end latency for online chatbots by up to 1.92×.
By Haoyu Zheng, Fangcheng Fu, Binhang Yuan, Yongqiang Zhang, Liang Deng, Hao Wang, Yuanyuan Zhu, Xiao Yan, Jiawei Jiang
FedFit introduces a federated fine‑tuning framework for large language models that reduces communication overhead by using a disjoint shared vector‑bank parameterization to reconstruct adapter matrices from two compact global vector banks. It resolves the aggregation dilemma between Sum‑of‑Products and Product‑of‑Sums through an alternating optimization schedule that alternates between accurate single‑bank updates and joint updates corrected by a Residual Spectral Aggregation mechanism. The method also incorporates blockwise quantization with client‑side error feedback and provides theoretical convergence guarantees, achieving perplexity comparable to standard federated LoRA while delivering up to 100× higher compression ratios on Qwen2.5 models.
By Hang Zou, Chao Zhang, Yuzhi Yang, Yu Tian, Samson Lasaulce, M\'erouane Debbah
CrossGMN introduces a graph metanetwork that processes a trained source network and an initialized target network simultaneously, enabling equivariant cross‑architecture weight‑space transformations. By preserving symmetry through cross‑network message passing, CrossGMN can refine target network initializations while remaining invariant to source permutations and equivariant to target permutations. Experiments demonstrate that CrossGMN accelerates knowledge distillation, transfers across datasets without retraining, and unifies compression from diverse source architectures into a common target architecture.
By Adir Dayan, Yam Eitan, Haggai Maron
arXiv:2610. 02191v1 Announce Type: new Abstract: While Large Language Models (LLMs) have demonstrated striking capabilities on frontier mathematical problems, it remains unclear whether they possess the structural mathematical understanding underlying their solutions.
By Shuo Xing, Zilin Dai, Chengyuan Qian, Fangzhou Lin, Wenjing Chen, Ping He, Pan Lu, Alvaro Velasquez, Mohit Bansal, Zhengzhong Tu
AURAL is a speech language model that performs adaptive latent reasoning by modeling multiple plausible reasoning continuations in latent space and jointly predicting chunks of future states, thereby reducing sequential forward passes and latency. The authors introduce a large bilingual dataset, AuralReason-683K, containing concise chain‑of‑thought annotations for emotion recognition, empathetic dialogue, and general reasoning, and use reinforcement learning (AURAL‑RL) to reward concise, high‑quality reasoning that adapts to problem difficulty. Experiments on two backbones show that AURAL‑RL matches or exceeds chain‑of‑thought reinforcement learning while achieving significant latency reductions, such as an 11.8× speed‑up on Qwen2.5‑Omni.
"whyItMatters":"The work demonstrates that latent reasoning can match the performance of explicit chain‑of‑thought methods while dramatically cutting response time, addressing the trade‑off between intelligence and speed in speech language models."
By Yuxiang Wang, Kunyu Feng, Yuancheng Wang, Zihang Liu, Shengbo Cai, Qinke Ni, Wan Lin, Tao Feng, Yingda shen, Ming-Hao Hsu, Zhixian Zhao, Liqiang Zhang, Teddy Sun, Steve Yves, Zhizheng Wu
Varda‑single‑1.0 is a medium‑range, data‑driven weather prediction system designed for Switzerland’s Alpine region, delivering hourly deterministic forecasts at 1 km resolution and global forecasts at 31 km. It uses two independently trained stretched‑grid Graph Transformer models—an autoregressive forecaster and a temporal downscaler—trained through a curriculum that starts with ERA5 reanalysis, proceeds to a 20‑year regional reanalysis, and ends with fine‑tuning on operational analyses. Over a one‑year verification, Varda‑single matches or surpasses MeteoSwiss’s numerical weather prediction baselines for most key metrics, though it underestimates local wind peaks and produces smoother convective precipitation fields.
whyItMatters":"The system demonstrates that high‑resolution machine‑learning models can rival traditional numerical weather prediction in complex terrain, offering a complementary tool for operational forecasting and research."
By Alberto Pennino, Francesco Zanetta, Michele Cattaneo, Claire Merker, Radi Radev, Jonas Bhend, Louis Frey, Hugues de Laroussilhe, Oph\'elia Miralles, Carlos Osuna, Daniele Nerini, Andreas Pauling, Daniel Hupp, Ulrich Hamann, Mary McGlohon, Marti Bosch, Luca Lanzilao, Marco Arpagaus, Lukas Jansing, Daniel Leuenberger, Mark A. Liniger, Katrin Ehlert, Matthew Chantry, H\aa{}vard Homleid Haugen, Gert Mertes, Ana Prieto Nemesio, Mario Santa Cruz, Jasper Wijnands, Gabriel Moldovan, Harrison Cook, Oliver Fuhrer