← Back to all news
arXiv Machine Learning By Max Henning H\"oth, Kristian Kersting, Bj\"orn Deiseroth, Letitia Parcalabescu

AtManRL: Towards Faithful Reasoning via Differentiable Attention Saliency

Read the original on arXiv Machine Learning →

AtManRL introduces differentiable attention manipulation to train salient chain-of-thought reasoning in large language models. By optimizing an additive attention mask that identifies the tokens necessary to recover the model’s answer, the method derives a saliency reward combined with an outcome reward in a GRPO framework with decoupled advantages. Across three models and four benchmarks, AtManRL improves saliency and faithfulness metrics while maintaining comparable accuracy to length‑penalty baselines.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Machine Learning
Jun 9

Attention Illuminates LLM Reasoning: The Preplan-and-Anchor Rhythm Enables Fine-Grained Policy Optimization

arXiv:2510. 13554v2 Announce Type: replace-cross Abstract: The reasoning pattern of Large language models (LLMs) remains opaque, and reinforcement learning (RL) typically applies uniform credit across an entire generation, blurring the distinction between pivotal and routine steps.

By Yang Li, Zhichen Dong, Yuhan Sun, Weixun Wang, Shaopan Xiong, Yijia Luo, Jiashun Liu, Han Lu, Jiamang Wang, Wenbo Su, Bo Zheng, Junchi Yan
llmsreinforcement-learning
More like this →
arXiv Computer Vision
Oct 23

RECAP: Looking Once Is Not Enough for Vision-Language Reasoning

The paper introduces RECAP, a training method that enhances vision‑language models by incorporating temporally localized reactivation of image evidence—called visual revisit—into the reinforcement learning process. Visual revisit peaks are shown to predict correctness and causally improve performance; masking them hurts accuracy more than masking non‑peaks, while adding revisit peaks to failed reasoning partially restores correct answers. RECAP applies revisit‑conditioned advantage estimation, dynamic discounting, and gated value prediction to propagate final rewards through visually important reasoning steps, achieving significant gains on HallusionBench and other benchmarks without extra rewards or inference‑time changes.

By Zhaolu Kang, Tailong Luo, Chenxin Li, Yingjie He, Zhenyu Yu, Fengyu Zhou, Jiachen Qian, Lei Wei, Shuang Chen, Jiachen Li, Eric Hanchen Jiang, Rongchao Zhang, Zhengtao Yao, Hoi Leong Lee, Guansu Wang, Kaiyue Zhou
llmsreinforcement-learningfine-tuningmultimodalbenchmarks
More like this →
arXiv Computer Vision
Oct 16

Perception or Reasoning? Diagnosing and Tackling Reinforcement Learning Bottlenecks in VLMs

The paper introduces Perception and Reasoning Attribution (PARA), a method that quantifies how much of a vision‑language model’s errors can be fixed by better perception versus stronger reasoning. PARA reveals that reinforcement learning (RL) benefits more from improving reasoning than perception, and motivates a new group‑relative attention distillation objective called GRAD. GRAD directly guides weaker rollouts toward the visual attention patterns of the best rollouts, consistently boosting performance across multiple model scales and benchmarks.

By Zeyuan Yang, Zhuohao Chen, Jiaben Chen, Yao Zhang, Maohao Shen, Chuang Gan
llmsreinforcement-learningefficiencymultimodalbenchmarks
More like this →
arXiv Computation and Language
Sep 1

Unlocking Fine-Grained Translation Quality Estimation in LRMs through Mutually Boosting Implicit and Explicit Reasoning

arXiv:2605.31378v2 Announce Type: replace Abstract: Large Reasoning Models (LRMs) still struggle with fine-grained translation quality estimation (QE), even with long reasoning chains. We argue that...

By Renfei Dang, Xinye Wang, Zhejian Lai, Weilu Xu, Shimin Tao, Daimeng Wei, Min Zhang, Shujian Huang
reinforcement-learningfine-tuning
More like this →