arXiv AI
Logit Leakage through Floating-Point Bias Addition
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Jul 17
Smarter and Cheaper at Once: Byte-Exact KV-Cache Grafting Turns a Frozen Small Model into a Verified-Knowledge Flywheel
arXiv:2607. 14431v1 Announce Type: cross Abstract: We report a way to make a frozen small language model both more capable and dramatically cheaper at once, without changing any weights.
arXiv Machine Learning
Jul 31
Subtract or Replay? Exact Deletion from Language-Model Memory
arXiv:2607. 27539v1 Announce Type: new Abstract: Exact deletion from persistent language-model memory depends on how that memory represents a record.
arXiv Computation and Language
Oct 19
Watermarkability: Measuring and Improving Language-Model Compatibility with Logit-Based Watermarks
The paper introduces "watermarkability," a metric that quantifies how compatible a language model is with a logit‑based watermark that boosts the probability of a specified token set during decoding. By measuring the expected minimum KL divergence needed to achieve a target probability mass, the authors can assess and compare models and watermark rules. They propose a two‑stage approach—first training a low‑rank adapter conditioned on the target set, then applying a minimal uniform logit bias—to reduce this cost, achieving significant perplexity reductions across multiple watermark schedules on OPT‑125M and Qwen3‑8B.
arXiv Machine Learning
Jul 3
PARTREP: Learning What to Repeat for Decoder-only LLMs
arXiv:2607. 01792v1 Announce Type: cross Abstract: While decoder-only LLMs excel at a vast array of natural language tasks, it suffers from an asymmetric information flow induced by causal attention: later tokens are richer in contextual grounding than earlier ones.
arXiv Machine Learning
Sep 14
Breaking the Token Ceiling: Distilling Smaller, Stronger Byte Models
The paper investigates whether small models distilled from larger ones behave similarly when using byte versus token tokenization. It introduces two methods—Marginalize‑It (approximate) and End‑Of‑Token (exact)—to convert token logits to byte logits, and conducts a large‑scale study on decoder‑only dense transformers ranging from 1 billion to 1 trillion bytes of data. Results show that while token‑based models excel early, byte‑based models eventually surpass them with more compute, achieving higher performance ceilings, greater data efficiency, and lower logit storage costs.
arXiv AI
Aug 3
Deconstructing Off-Policy Ratios: Entropy-Scaled Trust Regions for Asynchronous Reinforcement Learning
arXiv:2607. 22186v2 Announce Type: replace Abstract: Asynchronous reinforcement learning (RL) accelerates large language model (LLM) post-training by overlapping rollout generation with policy optimization, but the resulting stale, off-policy data can destabilize optimization and ultimately cause policy collapse.