arXiv:2609.08618v1 Announce Type: new
Abstract: Benchmark scores describe what a checkpoint can do now, but they do not determine how it will respond to the next training episode. We measure this mis...
By Zhongxuan Liu, Sicheng Zhou, Hongzhi Wang
The paper introduces Amplitude Gating (AG), an inference-time technique that selectively masks feed‑forward network (FFN) activation coordinates while keeping pretrained weights unchanged. AG employs six SwiGLU sites to separate shared and gate‑only inputs, expands gate/value channels, and uses magnitude‑ranked entropy masks or a Newton‑Schulz windowed ranking variant to provide alternative forward paths without retraining. Experiments on three Qwen checkpoints and eight text/tool endpoints show modest improvements in function‑call and ToolBench endpoints, though statistical significance is not established and further experiments are required.
By Sheng Xu, Zhen Chen, Junhua Wang, Boyuan Huang, Ke Jia, Jiadun Zhu, Yiming Xu
arXiv:2610.27724v1 Announce Type: new
Abstract: Dense linear probes assess task readability from language model (LM) representations, but do not directly reveal how many features suffice for predicti...
By Kaiyang Wan, Yuxia Wang
arXiv:2608.21766v1 Announce Type: cross
Abstract: Both capability and safety benchmarks rest upon the assumption that the behavior of language models undergoing a test is informative about their beha...
By Farzaneh Heidari, Amin Memarian, Guillaume Rabusseau
The paper investigates how post‑hoc spectral smoothing and spiking interventions on transformer feed‑forward projection matrices affect large language models. By altering singular‑value spectra while preserving singular vectors and the geometric mean, the authors evaluate the impact on a range of tasks—including reasoning, truthfulness, hallucination, safety, and personality probes—across several Llama and Qwen models. The results reveal that structured perturbations can selectively suppress or enhance specific capabilities, suggesting a lightweight method for probing how different abilities are organized and traded off within these models.
By Qince Luan, Jianhui Xu, Yang Xu
arXiv:2609.39929v1 Announce Type: cross
Abstract: Long-context failures of RoPE-based language models can arise from RoPE's intrinsic tradeoff between maintaining stable token preferences and disting...
By Yuyang Wu, Yufeng Du, Hao Peng