arXiv:2606. 17399v1 Announce Type: cross Abstract: When small transformers grok modular multiplication, prior work reports that the learned embedding has a "dense" Fourier spectrum requiring all frequencies.
By Huu Danh Nguyen (Stanford University)
arXiv:2506. 04985v2 Announce Type: replace Abstract: Large language models (LLMs) require substantial compute, and thus energy, at inference time.
By Boris van Breugel, Yelysei Bondarenko, Paul Whatmough, Markus Nagel
arXiv:2604. 13082v2 Announce Type: replace-cross Abstract: Grokking in transformers trained on algorithmic tasks is characterized by a long delay between training-set fit and abrupt generalization, but the source of that delay remains poorly understood.
By Laura Gomezjurado Gonzalez
arXiv:2606. 23044v2 Announce Type: replace-cross Abstract: Numbers have algebraic structure that standard neural embeddings often fail to expose.
By Hyunsang Hwang, Suhyun Bae, Donghun Lee
arXiv:2608. 10010v2 Announce Type: replace Abstract: Low-precision formats usually optimize scalar fidelity while inheriting conventional product arithmetic.
By Ye Qiao
The study investigates how post‑training quantization (PTQ) affects proactive interference (PI) in large language models. Using bitsandbytes, the authors compare FP16, INT8, and INT4/NF4 precision across three instruction‑tuned models and find that INT4 quantization markedly degrades accuracy under high interference, with INT8 also incurring a smaller penalty in two of the three models. The degradation is linked to increased same‑key intrusion errors and originates in the quantized transformer backbone rather than the output layer.
By Shayan Shahrabi-Farahani (Shahid Beheshti University, Tehran, Iran), Dara Rahmati (Shahid Beheshti University, Tehran, Iran)