The paper introduces algebraic integer number formats (AINF) as a framework for highly configurable and accurate compressed computation, emphasizing its reduction to purely integer arithmetic for error‑free multiply‑accumulate operations. A matrix multiplication engine leveraging AINF achieves performance close to the most efficient numerical counterparts on modern hardware accelerators. The authors further propose algebraic integer quantization (AIQ) as a lossy compression format for large language model inference, demonstrating competitive accuracy against the industry‑standard MX format at equivalent bit‑per‑parameter budgets, and suggest that higher‑order algebraic integers could yield even greater computational benefits.
By Anthony Kostal-Vazquez, Vassil Dimitrov, Branden Butler
arXiv:2604.13082v3 Announce Type: replace-cross
Abstract: Large language models today still make mistakes on multi-digit arithmetic when they do not have tools, and part of the reason is how numbers...
By Laura Gomezjurado Gonzalez
The study investigates whether large language models (LLMs) can generalize arithmetic reasoning across different input formats, such as numeric and verbal expressions in English, Spanish, and Italian. By using attribution patching, the researchers identified the specific internal circuits each model employs for numeric versus verbal arithmetic tasks. They found that the degree of overlap between a model’s numeric circuit and its verbal circuit predicts how well the model generalizes to verbal formats, explaining variations in performance across languages and formats without requiring labeled data.
By Andrea Gregor de Varda, Sana Pandey, Pengrui Han, Jacob Andreas, Evelina Fedorenko
arXiv:2608. 04048v1 Announce Type: cross Abstract: Serving large language models (LLMs) under diverse deployment constraints requires flexible trade-offs between accuracy, memory footprint, and throughput.
By Yu Luo, Bo Dong, Wenhua Cheng, Haihao Shen
arXiv:2607. 07066v1 Announce Type: cross Abstract: Transformers have demonstrated a remarkable ability to learn algorithmic reasoning, yet mechanistic analyses have mostly focused on globally invertible operations such as cyclic addition and group composition.
By Zitong Andrew Chen, Junaid Hasan, Akhil Srinivasan, Hemkesh Bandi, Jarod Alper
arXiv:2608. 13129v1 Announce Type: new Abstract: Large language models (LLMs) achieve strong results on mathematical reasoning benchmarks yet remain unreliable on elementary numerical tasks, including magnitude comparison, large-integer arithmetic, fractions, and scientific notation.
By Aoxin Ni
The paper introduces Fractional Power-of-Two (FPoT) quantization, a base‑√2 weight grid that enables 4‑bit weight quantization for large language models while keeping a fully integer shift‑add datapath. FPoT combines a fractional PoT grid with uniform integer activations, a calibration‑time grid‑alignment refinement, and a lightweight 1‑adder √2 approximation to reduce gate count compared to a 4‑bit multiplier. On Llama‑3‑8B, FPoT boosts zero‑shot commonsense reasoning by 8.72% over the standard PoT baseline and stays within 0.4% of uniform integer performance, with benefits extending across seven models and three bit‑widths.
By Sunghyun Wee, Geunjae Choi, Hyeonjin Kim, Suyoung Kim, Nojun Kwak
arXiv:2506. 04985v2 Announce Type: replace Abstract: Large language models (LLMs) require substantial compute, and thus energy, at inference time.
By Boris van Breugel, Yelysei Bondarenko, Paul Whatmough, Markus Nagel
arXiv:2607. 10855v1 Announce Type: new Abstract: Quantization is a powerful strategy to build capable and resource-efficient large language models (LLMs) by reducing the bitwidth of the parameters.
By Sirine Ayadi, S\'andor Dar\'oczi, Stephan G\"unnemann, Bertrand Charpentier
arXiv:2606. 15652v1 Announce Type: new Abstract: 4-bit quantization significantly reduces the memory footprint and accelerates the inference of large language models (LLMs).
By Yangjia Hu, Haodong Wang, Zicong Hong, Qianli Liu, Quanxin Shou, Jian Lin, Song Guo, Xiaowei Shen, Xiangjun Huang, Dian Wang, Jian Yang
arXiv:2510. 26690v3 Announce Type: replace Abstract: Low-Rank Adaptation (LoRA) has become a popular technique for parameter-efficient fine-tuning of large language models (LLMs).
By Amir Reza Mirzaei, Yuqiao Wen, Yanshuai Cao, Lili Mou
Large language models (LLMs) achieve strong results on mathematical reasoning benchmarks yet remain unreliable on elementary numerical tasks, including magnitude comparison, large-integer arithmetic, fractions, and scientific notation. This survey examines basic numerical understanding as a capability distinct from high-level mathematical reasoning.