arXiv AI
Running Fewer Loops Removes Refusal Behaviour in Recurrent-Depth Language Models
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv Machine Learning
Oct 3
Refusal Localizes, the Damage Relocates: Safety Layers Under Few-Sample Fine-Tuning
The paper investigates how fine‑tuning large language models with a small number of harmful examples can erode their refusal behavior, and explores whether localizing safety‑related behavior to specific layers or directions can provide robust defenses. Experiments across six checkpoints from four model families show that harmful and benign prompts remain linearly separable after attack, and that patching clean hidden states or freezing layers up to a transition depth can restore refusal. However, attackers can bypass these defenses by spreading updates or targeting singular directions, indicating that adaptive fine‑tuning can defeat localized repairs and highlighting the need for multiple defensive checks.
arXiv AI
1d ago
LEV: Thinking Fast and Deep with Looped Decision Models
LEV is a decision model that performs reasoning by looping its layers instead of generating text, allowing it to deepen its computation on a per-question basis. Trained jointly across all loop steps, a single LEV checkpoint can answer at any depth, improving accuracy—especially on hard questions—while keeping inference fast. On benchmark tests, LEV outperforms dense and cascade models by 5.4 to 8.8 accuracy points with sub‑50 ms latency, and on‑demand depth selection saves 15–24% of computation for minimal accuracy loss.
arXiv Machine Learning
2d ago
Learning to Compute Is Not Learning to Retain or Reuse in Looped Transformers
arXiv:2610.19768v1 Announce Type: new Abstract: Looped transformers compute by applying one block repeatedly, so they can run for more loops than they were trained with. Recurrent reasoners are known...
arXiv AI
Jul 15
Adaptive Compute in Latent World Models: When Depth Helps, Hurts, or Doesn't Matter
arXiv:2607. 10203v2 Announce Type: replace-cross Abstract: Adaptive-compute world models -- early-exit or mixture-of-depths predictors that spend variable depth per step -- assume depth buys better predictions and can be routed adaptively.
arXiv AI
Jul 14
When Does Depth Survive Composition? Compute--Quality Regimes in Latent World Models
arXiv:2607. 10203v1 Announce Type: cross Abstract: Adaptive-compute world models -- early-exit or mixture-of-depths predictors that spend variable depth per step -- assume depth buys better predictions and can be routed adaptively.
arXiv Machine Learning
2d ago
Does Depth Survive Composition, and Can a Router Use It? Adaptive Compute in Latent World Models
The paper investigates whether deeper predictor blocks in adaptive‑compute world models truly improve rollout accuracy and whether a router can select the appropriate depth per step. Experiments on nine DeepMind Control tasks show that deeper predictors do not always yield better accuracy, and that a greedy oracle selecting depth per step can reduce error by up to 37% compared to fixed‑depth mixtures, yet no trained router achieves this advantage. The study also finds that supervising early exits only at the first rollout step can mitigate depth‑inversion issues observed in some tasks.