Hugging Face Trending Papers

Chain of Faith: Policy-Internal Rewards for Reasoning Reliance

Read the original on Hugging Face Trending Papers →

The paper investigates how to make language models rely more on their own chain-of-thought reasoning when answering questions. It introduces four lightweight reward signals—gradient sensitivity, confidence gaps, progressive confidence gains, and reasoning–prompt swaps—added to group relative policy optimization. Experiments on two small instruction‑tuned models show that some reward configurations improve reliance on reasoning edits, though effects on accuracy and robustness vary, and high reward weights can lead to answer copying.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at Hugging Face Trending Papers.

arXiv Machine Learning
Sep 21

Rewarding Efficient Reasoning Improves Abstention on Underspecified Tasks in Reasoning Models

The paper introduces a new reward, GRPO, that encourages large reasoning models (LRMs) to efficiently determine whether a task is solvable before generating a full chain of thought. Fine‑tuning 4B LRMs with this reward improves their ability to abstain from answering unanswerable prompts by an average of 12.8% while producing 44% shorter chains of thought. The approach also preserves the models’ overall answering performance.

By Polina Tsvilodub, Max H\"oth, Michael Franke, Bj\"orn Deiseroth, Carina Kauf
arXiv AI
Oct 16

Learning to Reason with Selective Internal Feedback

The paper introduces SIF, a reinforcement learning technique that uses selective internal feedback to improve language models’ reasoning without external verifiers. SIF constructs comparative rewards by comparing token‑level self‑certainty across multiple windows and peers, rewarding only when both voting levels agree, and applies sample and group‑level filtering to decide which responses influence policy updates. Experiments on three backbone models across 12 mathematical and scientific benchmarks show that SIF’s final checkpoints outperform the best‑validation checkpoints of Intuitor and Vigor, achieving a 4.71‑point accuracy gain on Qwen2.5‑Math‑1.5B and maintaining accuracy during continued optimization with minimal test‑time rollouts.

By Xun Lu, Shumeng Yang, Xinrui Wang, Songjun Tu, Linjing Li, Daniel Dajun Zeng