arXiv AI

Language as a Wave Phenomenon: Semantic Phase Locking and Interference in Neural Networks

arXiv:2512. 01208v5 Announce Type: replace-cross Abstract: In standard Transformer architectures, semantic importance is often conflated with activation magnitude, obscuring the geometric structure of latent representations.

arXiv AI

Jul 10

ResonatorLM: Causal Resonant Field Mixing for Efficient Long-Context Language Modeling

arXiv:2607. 05583v2 Announce Type: replace-cross Abstract: Contemporary language models are dominated by the transformer architecture, which leverages self-attention mechanisms to enable more efficient, parallelized training across a wide set of documents and corpora.

By Archie Chaudhury

llms

arXiv AI

Jul 8

ResonatorLM: Causal Resonant Field Mixing for Efficient Long-Context Language Modelin

arXiv:2607. 05583v1 Announce Type: cross Abstract: Contemporary language models are dominated by the transformer architecture, which leverages self-attention mechanisms to enable more efficient, parallelized training across a wide set of documents and corpora.

By Archie Chaudhury

llms

arXiv AI

Jul 10

The Phasor Transformer: Resolving Attention Bottlenecks on the Unit Circle

arXiv:2603. 17433v2 Announce Type: replace-cross Abstract: Transformer models have redefined sequence learning, yet dot-product self-attention introduces a quadratic token-mixing bottleneck for long-context time-series.

By Dibakar Sigdel

llms benchmarks

arXiv AI

Jun 9

Component Ablation for Efficient Hybrid Language Model Architectures: Performance, Resilience, and Compression Implications

arXiv:2603. 22473v2 Announce Type: replace-cross Abstract: Hybrid language models combine softmax attention with linear-time sequence mechanisms such as state-space or linear-attention layers, but the functional contribution of each component type remains insufficiently characterized.

By Hector Borobia, Elies Segu\'i-Mas, Guillermina Tormo-Carb\'o

llms benchmarks

arXiv Machine Learning

Jun 29

Prism Transformer: Progressive Head Schedules for Hierarchical Attention Processing

arXiv:2606. 27449v1 Announce Type: new Abstract: Multi-head attention conventionally partitions the hidden dimension equally across all heads at every layer, enforcing an identical representational subspace dimension (dh = dmodel/h) throughout the models depth.

By Shubham Aggarwal

llms benchmarks

arXiv Machine Learning

Jun 2

Resonant Context Anchoring: Decoupling Attention Routing and Signal Gain at Inference Time

arXiv:2606. 01923v1 Announce Type: cross Abstract: Large Language Models (LLMs) frequently exhibit "contextual disregard" when faced with input evidence that conflicts with their internal parametric memory, leading to persistent factual hallucinations.

By Mingkuan Zhao, Yide Gao, Wentao Hu, Suquan Chen, Tianchen Huang, Zhenhua An, Zetao Chang, Xiayu Sun, Yuheng Min

llms efficiency safety

arXiv AI

Jun 16

Long-Context Modeling via GSS-Transformer Hybrid Architecture with Learnable Mixing

arXiv:2606. 16093v1 Announce Type: cross Abstract: Modeling long-range dependencies remains a central challenge in natural language processing.

By Kuzey Torlak, H\"useyin Arda Arslan, An{\i}l Dervi\c{s}o\u{g}lu, Beyza Nur Deniz, Onur Boyar

llms nlp efficiency

arXiv Machine Learning

Jun 5

SpanNorm: Reconciling Training Stability and Performance in Deep Transformers

arXiv:2601. 22580v2 Announce Type: replace-cross Abstract: The success of Large Language Models (LLMs) hinges on the stable training of deep Transformer architectures.

By Chao Wang, Bei Li, Jiaqi Zhang, Xinyu Liu, Yuchun Fan, Linkun Lyu, Xin Chen, Jingang Wang, Tong Xiao, Peng Pei, Xunliang Cai

llms

arXiv Machine Learning

Jun 19

GEMS: Geometric Constraints Enable Multi-Semantic Superposition in LLMs

arXiv:2606. 19946v1 Announce Type: cross Abstract: Activation steering controls model behavior by modifying intermediate hidden states at inference time without retraining.

By Yu Deng

llms benchmarks

arXiv AI

Jun 24

WAND: Windowed Attention and Knowledge Distillation for Efficient Autoregressive Text-to-Speech Models

arXiv:2604. 08558v2 Announce Type: replace-cross Abstract: Recent decoder-only autoregressive text-to-speech (AR-TTS) models produce high-fidelity speech, but their memory and compute costs scale quadratically with sequence length due to full self-attention.

By Hanna Lee, Tan Dat Nguyen, Jaehoon Kang, Kyuhong Shim

fine-tuning efficiency multimodal

arXiv Machine Learning

Jun 17

Olmo Hybrid: From Theory to Practice and Back

arXiv:2604. 03444v4 Announce Type: replace Abstract: Recent work has demonstrated the potential of non-transformer language models, especially linear recurrent neural networks (RNNs) and hybrid models that mix recurrence and attention.

By William Merrill, Yanhong Li, Tyler Romero, Anej Svete, Caia Costello, Pradeep Dasigi, Dirk Groeneveld, David Heineman, Bailey Kuehl, Nathan Lambert, Chuan Li, Kyle Lo, Saumya Malik, DJ Matusz, Benjamin Minixhofer, Jacob Morrison, Luca Soldaini, Finbarr Timbers, Pete Walsh, Noah A. Smith, Hannaneh Hajishirzi, Ashish Sabharwal

llms

arXiv AI

Jun 30

Emergence of Minimal Circuits for Indirect Object Identification in Attention-Only Transformers

arXiv:2510. 25013v2 Announce Type: replace-cross Abstract: Mechanistic interpretability aims to reverse-engineer large language models (LLMs) into human-understandable computational circuits.

By Rabin Adhikari

llms rag benchmarks safety

Related stories