arXiv AI
Aug 10
Ask-E: An Environment for Calibrated Question Generation
arXiv:2608. 06933v1 Announce Type: cross Abstract: Today, we improve models by training and evaluating them on problems at the frontier of their abilities.
LIFT: Scoped Constraints for Evaluating and Training Instruction Following
LIFT introduces Lift-Bench, a new instruction‑following benchmark that asks language models to draft a document based on a topic and a structured outline, with constraints scoped to specific sections for finer control. The benchmark includes both verifiable and judged constraints and a training counterpart, Lift‑Train, designed to reduce reward hacking by providing both main‑task and instruction‑following constraints. Experiments show that state‑of‑the‑art models struggle with the harder subsets, and training on Lift‑Train is less prone to reward hacking than traditional IF training.
Related stories
Hugging Face Trending Papers
Jul 30
IFHierBench: Hierarchical Instruction Following for Large Language Models
Instruction-following ability is critical for deploying large language models in real-world applications, where downstream components depend on the output satisfying specific constraints. Modern deployments increasingly handle the full task in a single LLM call, with one prompt specifying a layered output whose overall artifact, structural sections, and nested fields must each satisfy concrete constraints.
arXiv Machine Learning
Oct 15
Diagnosing and Correcting Reward Hacking in Verifiable Instruction Following
arXiv:2610.19525v1 Announce Type: new Abstract: Verifiable instruction-following benchmarks emphasize output-constraint satisfaction but can overlook whether responses meaningfully answer the underly...
Hugging Face Trending Papers
Oct 14
Diagnosing and Correcting Reward Hacking in Verifiable Instruction Following
Verifiable instruction-following benchmarks emphasize output-constraint satisfaction but can overlook whether responses meaningfully answer the underlying request, creating an evaluation blind spot. W...
arXiv AI
Jun 12
WildIFEval: Instruction Following in the Wild
arXiv:2503. 06573v3 Announce Type: replace-cross Abstract: Recent LLMs have shown remarkable success in following user instructions, yet handling instructions with multiple constraints remains a significant challenge.
arXiv Machine Learning
Jun 9
TinyJudge: Unverifiable Constraint Alignment via Lightweight Specialist Ensembles
arXiv:2606. 07520v1 Announce Type: cross Abstract: Instruction Following (IF) is a core capability of LLMs, requiring strict adherence to diverse constraints, ranging from verifiable ones (e.
arXiv Machine Learning
Aug 26
Mitigating Exploration Bias in RL for Multi-Instruction Following
arXiv:2608.23830v1 Announce Type: cross Abstract: RL has emerged as a powerful paradigm for enhancing the instruction following capabilities of LLMs. While existing training recipes achieve substanti...
arXiv Computation and Language
Oct 8
Constraint Tree Exploration for Learning from Language Feedback
The paper introduces TRACE, an algorithm that learns from natural‑language feedback by treating user intent as latent constraints and exploring feasible action regions organized in a tree. TRACE tests candidate constraints by generating compliant actions and only commits to a refinement when repeated feedback does not contradict it, offering two modes: falsification and identification. The authors provide theoretical coverage guarantees and demonstrate that TRACE‑Identification outperforms prompting baselines on six language‑feedback tasks, achieving higher success rates and robustness to identity corruption.
arXiv AI
Sep 18
MATCH: Model-Aware Tool Learning with Curriculum Scheduling and Hierarchically Gated Rewards
MATCH is a closed‑loop framework for model‑aware tool learning that combines curriculum scheduling with hierarchically gated rewards. It introduces Model‑Aware Curriculum Learning (MACL), which dynamically adjusts sample difficulty based on reward signals, and Hierarchical Tool‑call Gated Reward (HTGR), which allocates credit at the tool name, argument key, and argument value levels only when prerequisites are met. Experiments on API‑Bank and BFCL V3 show MATCH achieving 72.19% and 62.87% overall accuracy, outperforming both supervised and RL‑based baselines across multiple backbone models.
arXiv AI
Oct 5
Goldilocks RL: Tuning Task Difficulty to Escape Sparse Rewards for Reasoning
Goldilocks RL proposes an adaptive data‑selection strategy for reinforcement learning in language models, using a Selector network to predict reward variability and prioritize questions that are neither too easy nor too hard. By continuously adapting to the model’s evolving abilities, Goldilocks improves training efficiency on large‑scale reasoning datasets, achieving up to 78% fewer optimization steps compared to standard GRPO. The approach addresses the sample‑inefficiency problem of sparse rewards in reasoning tasks.
arXiv AI
Aug 12
Evaluation-Conditioned Training: Teaching Models to Generalize to Stronger Oversight Regimes
arXiv:2608. 10209v1 Announce Type: new Abstract: Feedback signals used to train Large Language Models (LLMs) are the primary driver of their behavior and our main lever for instilling alignment with human values and objectives.
arXiv Computation and Language
Sep 1
Small Reward Models via Backward Inference
arXiv:2602.13551v3 Announce Type: replace Abstract: Reward models (RMs) play a central role throughout the language model (LM) pipeline, particularly in non-verifiable domains. However, the dominant...