arXiv Machine Learning July 27, 2026 By Jinwoo Baek

Numerical Fragility in Transformers: A Layer-wise Theory for Risk Estimation and Selective Stabilization

Read the original on arXiv Machine Learning →

arXiv:2510. 21770v2 Announce Type: replace Abstract: Low-precision execution can induce substantial forward discrepancies in Transformers even for fixed weights and input, yet these discrepancies are usually monitored only at the output and lack a layer-wise theoretical account.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

arXiv AI

Jul 21

First-Order Predictable but Pairwise Fragile: Local Task Adaptation in Trained Transformers

arXiv:2607. 16821v1 Announce Type: cross Abstract: Task arithmetic, sequential fine-tuning, activation steering, and first-order random search all operate through relatively small perturbations around an already trained checkpoint, and they rely on different local approximations: individual perturbations should be first-order predictable, task updates should compose with controlled interference, useful tangent structure should be stable and possible to estimate, and weight edits should have counterparts in representation space.

By Irina Piontkovskaia, Sergey Nikolenko

llms fine-tuning

arXiv Machine Learning

Jul 21

An Adjoint-Sensitivity Framework for Lost-in-the-Middle Phenomena in Causal Residual Transformers

arXiv:2607. 17696v1 Announce Type: cross Abstract: We develop an adjoint-sensitivity framework for positional influence in causal residual Transformers and separate unconditional analytic results from conditional boundary-shape conclusions.

By Cheng Huan, Hongwei Yuan

llms safety

arXiv Machine Learning

Jul 21

Auditing Invisible Weight Updates with Reference Traces

arXiv:2607. 09800v2 Announce Type: replace Abstract: Master weights and stochastic rounding bypass invisible stored-weight updates but do not locate lost direct-storage proposals or parameters worth protecting.

By Zekai Shang

llms

arXiv Machine Learning

Aug 4

Round-Trip Consistency: Bidirectional Diffusion Models Can Predict Their Own Rollout Errors

arXiv:2608. 00675v1 Announce Type: cross Abstract: Autoregressive models accumulate error over long rollouts, yet at deployment there is no ground truth to measure it against.

By Alexander Scheinker

diffusion benchmarks

arXiv AI

Jul 15

Adaptive Compute in Latent World Models: When Depth Helps, Hurts, or Doesn't Matter

arXiv:2607. 10203v2 Announce Type: replace-cross Abstract: Adaptive-compute world models -- early-exit or mixture-of-depths predictors that spend variable depth per step -- assume depth buys better predictions and can be routed adaptively.

By Achyuthan Sivasankar

benchmarks

arXiv Machine Learning

Jun 19

Algebraic Dead Directions in LayerNorm Transformers: A Forward-Pass-Only Diagnostic at LLM Scale

arXiv:2606. 19491v1 Announce Type: new Abstract: Pretrained transformers sit near singular minima of the loss, where the Fisher information metric degenerates along dead directions: directions in parameter space along which the directional Fisher vanishes.

By Tejas Pradeep Shirodkar, P. J. Narayanan

llms

Related stories

First-Order Predictable but Pairwise Fragile: Local Task Adaptation in Trained Transformers

An Adjoint-Sensitivity Framework for Lost-in-the-Middle Phenomena in Causal Residual Transformers

Auditing Invisible Weight Updates with Reference Traces

Round-Trip Consistency: Bidirectional Diffusion Models Can Predict Their Own Rollout Errors

Adaptive Compute in Latent World Models: When Depth Helps, Hurts, or Doesn't Matter

Algebraic Dead Directions in LayerNorm Transformers: A Forward-Pass-Only Diagnostic at LLM Scale