The paper discusses how operator‑learning benchmarks are stored as full‑precision arrays that have grown to terabyte scale, and curators must decide how coarsely to compress them. It argues that the usual tolerance on reconstruction error of the stored field is measured in the wrong place, because what matters is the accuracy of a model trained on the compressed copy, not the raw field error. The authors propose a new metric: push a compressed field through a model trained at full precision and compare its output to the output on the clean field, which captures the portion of codec error that actually reaches the model’s output. This metric spans more than two orders of magnitude across six PDE families and correctly orders downstream cost where field error does not, reversing many cross‑dataset comparisons.
arXiv:2610.06095v2 Announce Type: replace Abstract: Operator-learning benchmarks are stored at full precision and have grown to terabyte scale. Rate-distortion theory says how many bits the stored fi...
arXiv:2611.06598v1 Announce Type: new Abstract: Low-Rank Adaptation (LoRA) adapts a frozen language model through a compact learned update, making it a natural setting for asking how much information...
arXiv:2608. 00675v1 Announce Type: cross Abstract: Autoregressive models accumulate error over long rollouts, yet at deployment there is no ground truth to measure it against.
The paper investigates speculative decoding for mixture‑of‑experts (MoE) language models, focusing on how deduplication of expert reads can reduce byte traffic and improve decoding speed. By modeling step time as a function of bytes requested and kernel bandwidth, the authors derive conditions under which speculation outperforms plain decoding and identify the optimal draft length as a staircase function of acceptance rate. Experiments on llama.cpp Metal, MLX‑LM, and llama.cpp CUDA show that deduplication is only faster for very short draft lengths (5–8 tokens) or not at all, and the predicted crossover points closely match measured savings across multiple configurations.
arXiv:2609.06008v1 Announce Type: cross Abstract: We present Cadence, an error-bounded lossy compressor for numeric time series pairing a 330M-parameter time-series foundation model (Google TimesFM-3...
The paper argues that the policy governing how tensors are rounded and reused during the backward pass—termed the backward‑state policy—is an integral part of the learning algorithm, not merely a memory or precision detail. Experiments with 390 M‑parameter models show that whether the backward pass reuses a forward’s rounded output or generates a new rounding can decisively affect training success, even when copy accuracy is high. The authors propose a method to determine, for each operator, which value should be read or substituted to preserve the correct gradient, and validate these predictions on PyTorch and Transformer Engine.
arXiv:2608. 12652v1 Announce Type: cross Abstract: Benchmark contamination is diagnosed today with n-gram overlap, with likelihood-based membership inference, or with canary strings, and each needs something usually unavailable: the training corpus, a well-chosen test statistic, or foresight at dataset release.
arXiv:2607. 13124v1 Announce Type: cross Abstract: Structured pruning is a hardware-friendly way to compress LLMs, but it is mostly validated on multiple-choice recognition tasks, while the same compressed checkpoints can collapse on the free-form generation that deployment actually requires.
The paper presents a certified rule that determines whether unseen data should be incorporated into a pretrained model’s context or its weights, based on an exact law for their information value. This law quantifies the code length saved on held‑out queries as the expected bits stored about the query, adjusted by a prior repair and decoding loss, and applies to any model, architecture, dataset, or task. The authors demonstrate that the rule’s ceilings correctly certify choices across diverse models—including transformers, TabPFN, occupancy networks, and stereo networks—and that an anytime‑valid test can reliably select the better approach (e.g., TabPFN over XGBoost or GPT‑2’s cache over fine‑tuning) with controlled error rates.
arXiv:2610.31935v1 Announce Type: new Abstract: PDE foundation models enable data-efficient adaptation but are deployed autoregressively, so short-horizon accuracy need not imply reliable simulation....
arXiv:2606. 25151v1 Announce Type: new Abstract: Physics-informed neural networks (PINNs) embed governing equations in their loss function, enabling mesh-free solutions to partial differential equations.
arXiv:2606. 05389v1 Announce Type: new Abstract: Lossy compression is essential for massive spatiotemporal data from scientific simulations.