← Back to all news
arXiv Machine Learning By Huy Hoang Le

Transmission Factors for Lossy Compression of PDE Training Data: Measuring What Reaches a Trained Operator

Read the original on arXiv Machine Learning →

The paper discusses how operator‑learning benchmarks are stored as full‑precision arrays that have grown to terabyte scale, and curators must decide how coarsely to compress them. It argues that the usual tolerance on reconstruction error of the stored field is measured in the wrong place, because what matters is the accuracy of a model trained on the compressed copy, not the raw field error. The authors propose a new metric: push a compressed field through a model trained at full precision and compare its output to the output on the clean field, which captures the portion of codec error that actually reaches the model’s output. This metric spans more than two orders of magnitude across six PDE families and correctly orders downstream cost where field error does not, reversing many cross‑dataset comparisons.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Machine Learning
Oct 28

Speculative Decoding for Mixture-of-Experts Models: Expert Bytes, Kernel Bandwidth and the Best Draft Length

The paper investigates speculative decoding for mixture‑of‑experts (MoE) language models, focusing on how deduplication of expert reads can reduce byte traffic and improve decoding speed. By modeling step time as a function of bytes requested and kernel bandwidth, the authors derive conditions under which speculation outperforms plain decoding and identify the optimal draft length as a staircase function of acceptance rate. Experiments on llama.cpp Metal, MLX‑LM, and llama.cpp CUDA show that deduplication is only faster for very short draft lengths (5–8 tokens) or not at all, and the predicted crossover points closely match measured savings across multiple configurations.

By Yuzheng Lan
llms
More like this →
arXiv Machine Learning
Oct 2

Backward-State Policy Is Part of the Learning Algorithm

The paper argues that the policy governing how tensors are rounded and reused during the backward pass—termed the backward‑state policy—is an integral part of the learning algorithm, not merely a memory or precision detail. Experiments with 390 M‑parameter models show that whether the backward pass reuses a forward’s rounded output or generates a new rounding can decisively affect training success, even when copy accuracy is high. The authors propose a method to determine, for each operator, which value should be read or substituted to preserve the correct gradient, and validate these predictions on PyTorch and Transformer Engine.

By Shuxiao Xie, Shuyang Xie, Dezhi Ran, Wei Yang, Tao Xie
llms
More like this →