arXiv:2606. 16222v1 Announce Type: new Abstract: Large Language Models (LLMs) increasingly rely on intermediate reasoning, yet explicit Chain-of-Thought (CoT) suffers from a linguistic space bottleneck: each thought must be decoded into tokens, causing high inference overhead.
arXiv:2610.05370v2 Announce Type: replace Abstract: Generative reward models (GRMs) are important for LLM optimization. Unlike scalar reward models, GRMs generate natural-language critiques alongside...
arXiv:2608.29168v1 Announce Type: new Abstract: The LLM-as-a-Judge paradigm has emerged as a scalable alternative to human evaluation. However, single-model judges are limited by their inherent model...
Rubric-based evaluation of large language models (LLMs) often assigns binary judgments to multiple criteria, but the certainty of these judgments varies widely. Existing aggregation methods treat all criteria as equally certain, potentially misleading reinforcement learning signals. The authors propose Uncertainty-aware Rubric Weighting (URW), a training-free framework that uses conformal calibration to quantify and weight criterion-level uncertainty, improving preference accuracy and downstream RL alignment.
arXiv:2608.30005v1 Announce Type: new Abstract: Rubric-based reinforcement learning extends RL beyond tasks with exact answers or rule-based verifiers by scoring responses against instance-specific c...
arXiv:2609.05910v1 Announce Type: cross Abstract: Reinforcement learning (RL) excels on tasks with verifiable rewards, but in open-ended tasks, the reliability of reward models remains a key challeng...
The paper introduces JudgeBiasBench, a benchmark that systematically quantifies judgment biases in large language model (LLM)-based judges across four dimensions and 12 bias types. It evaluates both generative and discriminative judges, revealing significant bias patterns that undermine reliability. The authors propose bias-aware training—reinforcement learning for generative judges and contrastive learning for discriminative judges—to reduce these biases while maintaining evaluation performance.
arXiv:2511. 19314v2 Announce Type: replace Abstract: Information-seeking is a core capability for AI agents, requiring them to gather and reason over tool-generated information across long trajectories.
arXiv:2602. 07533v2 Announce Type: replace Abstract: Reward models are critical for reinforcement learning from human feedback, as they determine the alignment quality and reliability of generative models.
arXiv:2606. 29481v1 Announce Type: cross Abstract: While reinforcement learning (RL) significantly enhances LLM reasoning, its efficacy is severely undermined by Pre-RL data overlap, where RL datasets overlap with pretraining or SFT corpora, causing models to exploit shortcuts by memorizing correct answers and fabricating post-hoc reasoning.
The paper introduces a method for creating query‑specific rubrics for DeepResearch‑style long‑form report generation by training rubric generators with reinforcement learning. It builds a dataset of queries annotated with human preferences, then uses a hybrid reward that includes preference consistency, format validity, and LLM‑based rubric evaluation. The learned rubrics outperform generic or manually constructed alternatives in distinguishing preferred reports and, when used as rewards, improve performance of both single‑agent and multi‑agent DeepResearch systems.
arXiv:2606. 09380v1 Announce Type: cross Abstract: Reinforcement learning with verifiable rewards (RLVR) has become a leading paradigm for improving the reasoning ability of large language models through outcome-based supervision.