The study investigates how quantization affects subliminal preferences in AI models, specifically a hidden "favourite animal: panda" preference implanted in Qwen2.5-7B-Instruct. While 8‑bit quantization preserves the preference, 4‑bit quantization produces divergent effects: Half‑Quadratic Quantization amplifies the preference to 64% panda probability, whereas NormalFloat suppresses it to 17%. The results show that the arrangement of rounding errors, not just their magnitude, drives these shifts, indicating that a 16‑bit audit cannot reliably predict real‑world behaviour under deployment‑level compression.
arXiv:2609.07901v1 Announce Type: new Abstract: Weight quantization largely determines the economics of serving open-weight LLMs. Its costs are usually assessed with capability benchmarks, on which 4...
The paper investigates how reducing numerical precision through model quantization impacts the vulnerability of neural networks to model inversion attacks. It provides theoretical bounds on mutual information changes and identifies data-dependent effects, especially at 4‑bit precision. Based on these findings, the authors propose a privacy‑aware post‑training quantization strategy that allocates bits adaptively, calibrates activation ranges, and jointly optimizes weight and activation scaling to improve inversion resistance while preserving model utility.
The study investigates how post‑training quantization (PTQ) affects proactive interference (PI) in large language models. Using bitsandbytes, the authors compare FP16, INT8, and INT4/NF4 precision across three instruction‑tuned models and find that INT4 quantization markedly degrades accuracy under high interference, with INT8 also incurring a smaller penalty in two of the three models. The degradation is linked to increased same‑key intrusion errors and originates in the quantized transformer backbone rather than the output layer.
arXiv:2609.06473v1 Announce Type: new Abstract: Inference-time activation steering enables behavioral control of large language models without parameter modification, while post-training quantization...
The paper investigates where post‑training quantization (PTQ) harms large language models (LLMs) and how to best allocate a limited precision budget. By causally raising each layer to 8‑bit precision across nine open‑weight models, the authors find that quantization damage is diffuse rather than concentrated in specific task circuits or weight statistics, and that globally refining quantization granularity outperforms selectively protecting the most recoverable layers. They also observe that the residual accuracy loss is budget‑limited and that peak recovery locations correlate with architecture within families but not across families.
arXiv:2608. 06564v1 Announce Type: new Abstract: Quantization is how large language models are actually deployed, and below four bits it is known to hurt.
The study evaluates how low‑precision compression affects adversarial robustness in EEG decoders used for brain‑computer interfaces. Using BCI Competition IV‑2a data, the authors compare 32‑bit floating‑point models (EEGNet and ShallowConvNet) with models pruned to 50 % and quantized to INT8 via post‑training quantization (PTQ) or quantization‑aware training (QAT). Results show that accuracy‑preserving compression does not improve direct robustness—PGD attack accuracy remains 22–24 % across all variants—yet pruning reduces bidirectional transfer efficiency more than PTQ, indicating that robustness, transferability, and deployment efficiency are distinct properties of compressed EEG decoders.
arXiv:2607. 08734v1 Announce Type: new Abstract: Post-training quantization is widely used to deploy large language models in resource-constrained settings, yet its evaluation relies almost exclusively on accuracy and perplexity.
arXiv:2606. 03002v1 Announce Type: cross Abstract: Quantization is a standard path to deploying large language models, and a quantized model is typically judged acceptable when its perplexity or downstream accuracy stays close to the full-precision original.
arXiv:2608.28809v1 Announce Type: new Abstract: Extreme low-bit inference offers a route toward smaller models and constrained deployment. Ternary language models restrict weights to $\{-1,0,+1\}$, a...
arXiv:2607. 24440v1 Announce Type: cross Abstract: Vision-language models (VLMs) deployed on consumer hardware must decide when to answer and when to defer, and that decision depends on having a confidence signal that tracks correctness.
arXiv:2607. 08779v1 Announce Type: cross Abstract: The signed integer alphabet contains one more negative representable value than positive.