← Back to all news
arXiv AI By Zhenzhen Ren, Xiaoqing Zhang, GuoBiao Li, Bowen Fang, Yanxi Wang, Xinpeng Zhang, Ke Han

Test-Time Confidence Adaptation under RL-Induced Trajectory Shift

Read the original on arXiv AI →

The paper investigates how trajectory-based confidence estimation for large language models deteriorates after agentic reinforcement learning (Agentic RL) due to shifts in reasoning trajectories. It discovers that more stable trajectories consistently correlate with correct answers, a relationship that remains intact across both base and RL models. Leveraging this insight, the authors introduce StaR-TTA, a stability-guided test-time confidence adaptation framework that learns from labeled base trajectories and applies stability-derived reliability rankings to unlabeled RL trajectories, achieving improved AUROC scores across multiple QA benchmarks.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv AI
Sep 12

Beyond Confidence: Stability-Aware Test-Time Adaptation for LLM Reasoning

The paper introduces TASCO, a test‑time adaptation framework that enhances Large Language Model reasoning by incorporating local stability into confidence‑based adaptation while keeping the model frozen. TASCO optimizes a lightweight task‑level prefix using two perturbation strategies—Random Perturbation for distributional stability and Sharpness‑Aware Perturbation for worst‑case sensitivity—to ensure that high confidence aligns with correctness. Experiments show that TASCO improves reasoning accuracy and token efficiency across various LLMs and benchmarks, and behavioral analyses confirm that it maintains stable confidence without over‑concentrating the predictive distribution.

By Bincheng Gu, Min Gao, Zongwei Wang, Yibing Bai, Yulan He, Junliang Yu
llmsbenchmarks
More like this →
Hugging Face Trending Papers
Sep 10

Beyond Confidence: Stability-Aware Test-Time Adaptation for LLM Reasoning

The paper introduces TASCO, a test‑time adaptation method that improves Large Language Model reasoning by optimizing stability‑aware confidence. It keeps the LLM frozen and uses a lightweight task‑level prefix, applying Random Perturbation for distributional stability and Sharpness‑Aware Perturbation for worst‑case sensitivity. Experiments show enhanced reasoning accuracy and token efficiency across various LLMs and benchmarks, while preserving stable confidence without over‑concentrating predictions.

llmsbenchmarks
More like this →
arXiv AI
Oct 5

Test-time Calibration Learning for Large Language Model Reasoning

The paper introduces Test-Time Calibration Learning (TTCL), a label‑free framework that adapts both reasoning accuracy and verbalized confidence of large language models directly on unlabeled target‑task data. TTCL generates self‑supervision signals from multiple model responses, enabling calibration without ground‑truth labels and proving theoretically as a bounded surrogate for the ideal calibration objective. Experiments on mathematical reasoning and factual question answering show consistent improvements, with base models gaining an average 40.13% in accuracy and 70.80% in ECE reduction across eight benchmarks, and further gains under domain shift.

By Zizhuo Zhang, Xiong Peng, Jingwei Sun, Rong Yao, Shixiong Kai, Mingxuan Yuan, Bo Han
llmsnlpreinforcement-learningbenchmarks
More like this →
arXiv AI
Jun 3

Exploiting Verification-Generation Gap: Test-Time Reinforcement Learning with Confidence-Conditioned Verification

arXiv:2606. 03608v1 Announce Type: cross Abstract: Test-time reinforcement learning has emerged as a promising paradigm for enhancing the complex reasoning abilities of large language models in a completely label-free manner.

By Jiahui Li, Jianfeng Shan, Wenpei Chen, Shunyu Wu, Jian Lou, Wenjie Feng, Dan Li, See-Kiong Ng
llmsreinforcement-learningbenchmarks
More like this →