The paper introduces the first systematic study of zero‑knowledge (ZK)‑friendly quantization for large language models (LLMs). It defines what makes a quantization scheme suitable for ZK proof generation and evaluates nine models, including Qwen2.5‑14B and Qwen3‑30B‑A3B, across various weight, activation, and nonlinear lookup precisions. Findings reveal that activation precision is more critical than weight precision, nonlinear lookup approximations can dominate utility loss, and that reducing bit‑width or lookup size does not always lead to proportional proving cost savings, highlighting the need for operator‑aware precision selection.
arXiv:2603. 18046v2 Announce Type: replace-cross Abstract: We present NanoZK, a zero-knowledge proof system for verifiable LLM inference: clients and third-party auditors check that a provider executed the advertised model on a committed input without learning weights or activations.
Large language models achieve strong performance on arithmetic reasoning benchmarks, and one common response to arithmetic brittleness is to delegate computation to code. Yet models are still often used in settings where they must reason directly from natural language, and trustworthy models should solve small-number arithmetic word problems without external tools.
arXiv:2606. 03606v1 Announce Type: cross Abstract: Large language models achieve strong performance on arithmetic reasoning benchmarks, and one common response to arithmetic brittleness is to delegate computation to code.
arXiv:2605. 08692v2 Announce Type: replace Abstract: Post-training weight-only quantization to 4 bits is widely used to reduce the memory and compute costs of large language model inference.
arXiv:2605. 17160v2 Announce Type: replace-cross Abstract: Model quantization is widely used to reduce memory, latency, and deployment cost, and is typically judged by whether predictive accuracy is preserved.
arXiv:2603. 13334v4 Announce Type: replace Abstract: Lipschitz-based robustness certification bounds a network's sensitivity through concrete numerical computation rather than symbolic reasoning, and so scales efficiently.
arXiv:2609.37887v1 Announce Type: new Abstract: Activation and key-value cache precision change what a quantized language model computes without altering its stored weights. Direct weight-code bounds...
arXiv:2607. 14181v1 Announce Type: cross Abstract: The growing adoption of local inference frameworks such as Ollama has made it increasingly common for developers to run large code models on laptops and other resource-constrained hardware.
SpecQuant is a training‑free framework that merges speculative decoding with multi‑parent quantization to enable adaptive, efficient inference of large language models. It generates several quantized variants (INT4, FP8, FP16) from a single base model and routes queries to the appropriate variant based on predicted complexity, using lightweight models for simple tasks and full‑precision models for complex reasoning. Evaluations on Qwen2.5 models across MMLU, AlpacaEval, and GSM8K show 35‑43% speedups with less than 2% accuracy loss, facilitating practical on‑device LLM deployment without specialized infrastructure.
arXiv:2507. 23035v4 Announce Type: replace Abstract: Large language models (LLMs) have demonstrated impressive capabilities across a wide range of applications, but demand substantial memory and compute resources during inference.
PANDA is a scalable system that uses zero‑knowledge proofs to certify the robustness and fairness of neural networks without revealing their private parameters. Built on the CROWN robustness framework, PANDA introduces a novel algorithm for proving linear relaxation bounds on non‑linear activation layers, producing lightweight proofs. The system can generate proofs for networks with over 2.9 million parameters in just five minutes and verify them in ten seconds, scaling polynomially with network size and enabling verification of models four orders of magnitude larger than prior ZKP‑based approaches.
arXiv:2607. 08779v1 Announce Type: cross Abstract: The signed integer alphabet contains one more negative representable value than positive.