The paper introduces AIMES, a framework for adaptive multi-value activation steering in large language models. AIMES builds layer‑specific bipolar directions for moral‑foundation values and uses intermediate‑layer vocabulary readouts as online observers to guide a controller that adjusts intervention strengths at each decoding step. Experiments across instruction‑tuned model families show that AIMES achieves depth‑dependent advantages over fixed joint steering and prompt‑based steering, with smaller activation‑space interventions and comparable response quality.
By Payel Bhattacharjee, Ravi Tandon
The paper introduces Q-Target Pretrained Transformers (QTPT), a method that replaces supervised behavior cloning with a Bellman-style Q‑target objective for in‑context reinforcement learning. QTPT retains the context‑conditioned Transformer architecture but learns to estimate action values using rewards and transitions from the context, rather than merely imitating offline actions. The authors provide theoretical analysis in stochastic linear bandits and finite‑horizon MDPs, demonstrating improved robustness to weak or suboptimal data, and empirically show gains over supervised pretraining on controlled RL benchmarks and extensions to D4RL Kitchen and AntMaze.
By Yichen Lin, Xuyuan Xiong, Xue Wang, Xiangfu Meng, Mike Mingcheng Wei, Tao Yao
The paper introduces TRACE, a fine‑tuning framework for Retrieval‑Augmented Generation (RAG) that addresses conflicts between retrieved knowledge and a model’s internal knowledge. TRACE uses multi‑agent debate traces to identify correct and incorrect candidates and answer‑shift patterns, providing fine‑grained supervision for reliable knowledge‑source selection. It also incorporates an answer‑completeness regularization mechanism to prevent empty, overly short, or prematurely terminated responses, thereby improving robustness against misleading retrieved content and enhancing answer quality.
By Zhengchen Huang, Yundong Sun, Minrui Song, Shuanglong Yao, Ye Liu, Ji Chen, Xing Wang
The paper presents a theory-informed computational framework that converts cross-disciplinary theories of fake news into measurable features for automated detection and explanation. By reviewing theories from social sciences, psychology, economics, and more, the authors establish a broad theoretical foundation for computational modeling. Experiments on benchmark datasets demonstrate that theory-derived features are predictive, provide interpretable diagnostic signals, and that multi-feature models generally outperform individual features, though gains are modest.
By Zhaoyang Cao, Miriam Metzger, Reza Zafarani
The paper introduces TalkMesh, a decentralized network of small language model agents that learn to communicate effectively during inference. Each agent proposes an answer, scores it with a confidence head, and the most confident agent broadcasts a hint; lower‑confidence agents revise their proposals if a new suggestion scores higher. This gossip‑based consensus, trained via group relative policy optimization, enables a mesh of three agents to match the accuracy of majority voting over 32 samples, and scales to larger meshes to significantly boost performance on benchmarks like GSM8K and MATH-500.
By Mehmet Kerem Turkcan
LUMO (Lightweight Unified Multilingual Orchestrator) is a privacy‑preserving offline voice assistant that runs entirely on edge hardware, specifically a Raspberry Pi 5 with 8 GB RAM. It integrates local ASR, a 4‑bit GGUF‑quantized LLM, and TTS to deliver end‑to‑end response latencies of 2.0–4.0 s, a 6.8 % WER on short English utterances, and lower peak power consumption (~9 W) compared to existing edge assistants. The system also supports Bangla speech, enabling multilingual use in low‑resource settings.
By Md. Mehedi Hasan Naeem, Mst. Kamrunnahar Ruma, Nafiza Anjum, Shakila Sultana, Md. Sujan Ali
The paper introduces a mechanism‑aware conditioning framework that uses a nudged coarse ensemble to capture local instability geometry in chaotic systems. By injecting ensemble covariance statistics via a small FiLM module, the authors enhance rare‑event emulation in both a low‑dimensional chaotic benchmark and a quasi‑geostrophic flow model, achieving significant improvements in exceedance‑frequency and tail‑density errors with limited data. The approach demonstrates that local instability information can be leveraged as a practical conditioning signal for data‑efficient emulation of extreme events.
By Isabella S. Thiel, Juan Bello-Rivas, Yannis G. Kevrekidis, Themistoklis P. Sapsis
CacheReforge is a method for recovering stale key‑value (KV) caches in large language models when lightweight adapters evolve. It represents stale caches as layer‑wise mixed‑version objects and uses adapter anchors, sensitivity calibration, drift accumulation, and restart boundaries to decide between direct reuse, bounded recomputation, or full suffix recovery. Experiments on Qwen2.5 models with continual LoRA updates show a 92.4% reduction in mean KL divergence while only recomputing 5.44% of layers and cutting cache‑maintenance time by 93.2% compared to full prefill.
By Yuhang Cao, Yanzhou Mu, Chunrong Fang, Zhenyu Chen
The paper investigates how momentum influences optimization in a river‑valley loss landscape, where a low‑loss manifold is surrounded by steep orthogonal directions. It shows that momentum stabilizes large learning rates that vanilla gradient descent cannot tolerate, enabling faster progress along the river. The study also finds that in very flat, slowly spinning rivers, momentum itself does not directly accelerate tracking, but the larger permissible learning rate does.
By Miao Lu, Zeyu Bian, Kaiyue Wen, Beining Wu, Siyu Chen, Tianhao Wang, Zhiyuan Li
The paper presents a self‑supervised approach to learning representations of auroral emission spectra using a 1D Vision Transformer trained with a masked autoencoder on 223,000 unlabelled spectra. The pretrained model recovers key emission‑line intensity ratios with high accuracy (R² = 0.91) and, with a single linear probe, matches expert‑designed feature classifiers. When fine‑tuned, it surpasses the previous supervised auroral classifier (macro‑AP 88.5 vs. 77.8) and achieves a 0.870 mAP, outperforming a model trained from scratch by 0.159 using only 10 % of the labels, while attribution reveals reliance on N₂⁺ bands.
By Matthieu Le Lain, Ga\"el Cessateur, S\'ebastien Lef\`evre
The paper investigates how different speech content representations—such as SSL features, supervised tokens, posteriorgrams, and neural audio codecs—perform when used to train a generative model that produces audio conditioned only on each representation. By evaluating the generated audio on content, speaker identity, and prosody, the study identifies two regimes: some representations almost fully reconstruct the original audio, while others effectively separate speaker identity. The findings reveal that disentanglement of speaker identity depends on both the training objective and the representation’s information capacity, rather than supervision alone.
By Diego Torres, Axel Roebel, Nicolas Obin
The paper introduces a low-cost method for detecting hallucinations in large language models by treating the model as a black-box dynamical system. It projects responses into a high-dimensional manifold, models the latent state-space dynamics with Koopman operator theory, and uses differential residual scores from transition operators to distinguish factual from hallucinated outputs. The approach requires only a single-sample pass and shows state-of-the-art performance across three benchmarks with reduced resource overhead.
By Dan Wilson, Mohamed Akrout
The paper introduces DUALLM, a dual-method pipeline that uses a Large Language Model and a fine‑tuned small language model to classify Linux kernel security patches with high precision. By analyzing commit titles, messages, diffs, and code context, DUALLM achieves 87.4% accuracy and an F1‑score of 0.875, outperforming existing methods. It successfully identified 111 recent patches addressing out‑of‑bounds or use‑after‑free vulnerabilities, with 90 confirmed true positives and proof‑of‑concept exploits demonstrating the validity of the classifications.
By Xingyu Li (UC Riverside), Juefei Pu (UC Riverside), Yifan Wu (UC Riverside), Xiaochen Zou (UC Riverside), Shitong Zhu (UC Riverside), Qiushi Wu (UC Riverside), Zheng Zhang (UC Riverside), Joshua Hsu (UC Riverside), Yue Dong (UC Riverside), Zhiyun Qian (UC Riverside), Kangjie Lu (UC Riverside), Trent Jaeger (UC Riverside), Michael De Lucia (UC Riverside), Srikanth V. Krishnamurthy (UC Riverside)
arXiv:2609. 31093v1 Announce Type: new Abstract: Scaling language models to long contexts is limited by the quadratic cost of self-attention.
By Bohao Tang, Zhen Qin, Yuqi Pan, Zheng Li, Pengfei Liu
The paper introduces the Phase-Coherent Transformer (PCT), a complex-valued architecture that replaces traditional softmax attention with a real-valued, smooth gate applied to L2-normalised query-key similarities. PCT eliminates token competition, preserving phase information across layers, and demonstrates strong generalisation on a variety of mid-scale benchmarks, outperforming both standard softmax Transformers and other complex-valued counterparts. Experiments confirm that the gate design is essential: preserving negatively aligned phase components is crucial for performance, while violating these conditions leads to degradation or collapse on long-range tasks.
By Leona Hioki
The paper introduces mentored decoding, a formal framework for lossy speculative decoding that can accelerate inference of autoregressive language models while potentially improving output quality. It connects this inference technique to boosting theory and extends it to all f‑divergences, revealing geometric insights for total variation, simple approximations tied to boosting compliance, and a divergence‑independent data structure enabling efficient optimal parameter queries and mentored distribution construction.
By Vivien Tran-Thien, Richard Nock
arXiv:2609. 30501v1 Announce Type: new Abstract: Although bilevel optimization (BLO) has emerged as a powerful framework for addressing many complex and nested machine learning problems in recent years, most existing studies are confined to the lower-level strongly convex (LLSC) or lower-level generally convex (LLGC) settings (i.
By Zhiyao Zhang, Menglu Yu, Alvaro Velasquez, Nathaniel D. Bastian, Jia Liu
The paper introduces the Progressive Memory Transformer (PMT), a transformer variant that adds writable, window‑aligned memory to expose mid‑range representations alongside token and sequence‑level outputs. PMT is trained with a hierarchical learning framework that applies separate objectives at local, mid‑range, and global scales, encouraging the model to capture fine‑grained variation, window‑level motifs, and overall sequence agreement. Experiments on seven UCR/UEA/UCI classification datasets, a cue‑retention probe, and forecasting tasks show that PMT achieves strong low‑label classification performance, competitive multi‑horizon forecasting, and evidence that its memory states encode mid‑range motifs.
By Tord Sture Stangeland, Andreas K\"ohler, Steffen M{\ae}land, Ad\'in Ram\'ires Rivera
Skill Profiling with Attributable Reasoning (SPAR) is a wearable system that uses an eight‑IMU garment and pressure‑insole sensors to classify boxing punches as expert or novice. It provides multi‑tier explanations—joint‑level attributions for analysts, counterfactual kinetic‑chain layers for coaches, and plain‑language narratives for athletes—to enable actionable feedback. In a study of 17 participants and 4,713 punches, SPAR achieved a leave‑one‑participant‑out AUC of 0.842, and thematic analysis of coach interviews identified six key feedback themes.
By Nibraas Khan, Hanchen David Wang, Enya Bullard, Ritam Ghosh, Ruj Haan, Aarav Agrawal, Meiyi Ma, Nilanjan Sarkar
The paper argues that for large‑context autoregressive language‑model inference, memory bandwidth—specifically the Key‑Value (KV) cache—becomes the limiting resource rather than arithmetic throughput. It analytically derives how arithmetic intensity decays with context length for NVIDIA H100, NVIDIA B200, and AMD MI300X, identifies crossover points where KV traffic overtakes weight traffic, and evaluates representative techniques across five compression domains. The study finds a three‑regime behavior: below the crossover, weight traffic dominates and KV compression offers little benefit; beyond it, KV traffic dominates and compression methods trade quality for bandwidth, with paging and prefix sharing being lossless but capacity‑limited, while quantization and eviction directly reduce bandwidth at the cost of accuracy.
whyItMatters":"The work provides a unified analytical framework and a standardized protocol that enable consistent comparison of KV‑compression techniques across hardware and workloads, guiding practitioners in selecting appropriate methods for long‑context inference."
By Tejinder Singh