LoRA, PEFT, instruction tuning and domain adaptation — adapting a pretrained model without paying to train one.
Language models are increasingly expected to support users in stressful or uncertain situations, where warmth and empathy are desirable. While recent work associates warmer models with reduced reliabi...
The paper reproduces five factuality alignment methods for large language models—three on‑policy RL approaches (PPO, GRPO, Dr.GRPO) and two offline preference optimization methods (DPO, KTO)—using a common benchmark and evaluation metrics. None of the methods consistently outperforms the base model, and the rewards appear to favor format, length, or brevity rather than factuality or relevance. The study also highlights problems with current evaluation practices, noting that metrics and benchmarks often fail to measure prompt relevance and rely on fragile LLM judges and retrieval for verification.
Agent harnesses require a low-latency guardrail model together with customizable audit policies. However, deploying an auto-regressive guardrail model remains challenging: such models incur high infer...
Diffusion language models (DLMs) enable flexible parallel generation by resolving tokens according to the model's evolving predictability rather than a fixed left-to-right order. This flexibility, how...
MERIT is a three‑stage framework that enables language models to reason over table images by first adapting a table‑specific encoder through reconstruction, localization, and color‑conditioned supervision. It then fine‑tunes the model with supervised evidence selection and reasoning, and finally refines the policy using reinforcement learning with distinct rewards for evidence correctness, calculation, final answers, and output format. The approach is evaluated on table question answering and fact verification, demonstrating the benefits of visual inputs and training components across diverse table structures and visual presentations.
Functional near-infrared spectroscopy (fNIRS) provides portable and spatially localized measurements of cortical hemodynamics, but differences in channel count, probe layout, signal distribution, and...
Medical multimodal reasoning models are typically trained on teacher-generated rationales, discarding the valid reasoning in students' own failed trajectories, while general-domain correction methods...
RAN-JEPA is an action-conditioned world model designed for energy-efficient mobile network control. It first learns a predictive latent representation from abundant observation-only telemetry, then adapts this representation to a target deployment domain and learns action-conditioned latent dynamics from limited controlled sleep/wake trajectories while keeping the encoder fixed. Evaluations on public operator data and a real-equipment 5G testbed show that RAN-JEPA outperforms a GPT-2 baseline and a mechanistic rule in prediction accuracy, produces coherent future trajectories under different sleep plans, and achieves higher sleep ratios in closed-loop control with comparable congestion costs.
Inference-time alignment steers diffusion models toward downstream objectives without costly fine-tuning, but existing methods typically optimize expected cost (or reward) without explicitly controlli...
Sampling multiple reports creates an inference problem: how should sampled evidence be converted into a final decision, and does a better structured decision yield a better report? We study these ques...
SWE-Critix is a post‑training pipeline that creates generalist verifiers for coding‑agent trajectories. It uses supervised fine‑tuning followed by reinforcement learning to train 30B models on CodeForge‑Preview data, enabling the verifier to predict completion labels and generate natural‑language explanations. The approach raises classification accuracy from 57% to 78% on a held‑out test set.
The paper introduces TerminalHorizon, an automated engine that synthesizes long‑horizon trajectories for terminal agents by reconstructing environments from real‑world professional work and progressively expanding tasks to require further exploration and improvement. Using this engine, the authors create 1.5K task environments and fine‑tune Qwen3.5‑27B, producing TerminalHorizon‑27B, which shows significant performance gains across 12 benchmarks in coding, agentic production tasks, and reasoning. Case studies demonstrate the model’s ability to experiment beyond initial solutions, refine methods based on feedback, and preserve effective improvements in final deliverables.
The paper introduces VIGOR, a system that enhances large language model–generated industrial scene programs by adding verification and staged geometric refinement to ensure engineering feasibility. It builds a dataset of 1,749 industrial scenarios with executable programs and 94 geometric checks each, and uses supervised fine‑tuning, local projection, and conflict‑graph repair to address violations. VIGOR achieves a 96.86% complete feasibility rate and outperforms baselines by 10.42 points in overall score.
RATED (Reliability‑Aware Tool Retrieval via Execution Dynamics) is a retrieval framework that blends robust semantic matching with dynamic reliability adaptation for AI agents using external tools. It uses a frozen dual‑encoder with contrastive fine‑tuning for semantic relevance and a lightweight contextual bandit module that re‑ranks tool candidates based on real‑time execution feedback, allowing rapid adaptation to runtime failures without costly retraining. The EcoAgent benchmark, comprising 5,172 real‑world tools and over 210,000 query‑tool pairs, shows that RATED improves mean reciprocal rank by 52.2% over the ToolLLM API retriever and cuts fault selection from 86.2% to 2.9% during severe API outages, with negligible latency overhead.
The paper investigates how language models can aid their own continual learning by generating replay data and selecting preservation targets. By organizing task descriptions into Task Banks, the authors show that replay generated by frozen models reduces forgetting on tasks outside the bank and that informed selection of replay targets can improve retention. The study demonstrates that models can support broader self‑preservation, especially when larger models are used, and that replacing new‑task examples with replay can enhance preservation under a fixed example budget.
ProgramGym introduces a scalable framework for training long-horizon program reproduction agents, comprising Program-Env and Program-Agent components. Program-Env builds verifiable training environments from repositories via a feedback loop that repairs sandbox builds and improves black-box tests, yielding 1,648 environments across C, C++, Go, and Rust. Program-Agent employs a role-isolated workflow of analysis, implementation, and review with revision management, boosting performance on ProgramBench and raising Qwen3.8-27B’s accuracy from 34.87% to 42.84% and surpassing flagship models with fine-tuning.
The paper introduces Explanatory World Models (EWM), an attribution framework that leverages explanations from a frontier language model to identify earlier events that plausibly contributed to a delayed outcome. EWM redistributes reward recursively while preserving the undiscounted episode return, and is evaluated on long‑horizon RL benchmarks, a public sequential‑recommendation task, and an industry‑scale generative recommender, consistently outperforming baseline methods, especially when rewards are highly delayed. The study demonstrates that structured explanations can provide reusable credit signals for learning from delayed outcomes.
The paper introduces UPTURN, a method that improves large language model self‑correction by learning where to intervene in a failed reasoning rollout and injecting a new user turn to redirect the model, rather than fine‑tuning the LLM. A small 130K‑parameter MLP head is trained on a frozen encoder using only final‑answer correctness to select intervention points. Experiments on four reasoners and six math benchmarks show that UPTURN consistently raises accuracy over a 32‑rollout baseline and outperforms existing self‑correction and reflection‑token injection techniques while transferring to new tasks without retraining.
The paper introduces CORE‑Seg, an end‑to‑end framework that combines reasoning and segmentation for complex lesion detection in medical images. It leverages a Semantic‑Guided Prompt Adapter and a progressive training strategy from SFT to GRPO, using an adaptive dual‑granularity reward mechanism to address reward sparsity. CORE‑Seg achieves state‑of‑the‑art performance, with a mean Dice score of 37.06%—14.89% higher than the next best baseline—and reduces the failure rate to 18.42%.
Sample-Adaptive Zeroth-Order Optimization (SAZO) is a memory‑efficient fine‑tuning method for large language models that learns sample importance online and allocates queries adaptively. It estimates importance using gradient‑norm information from directional differences and tracks evolving importance with per‑example exponential moving averages, requiring only O(N) additional scalar state. SAZO provably matches the leading convergence term of a minimum‑variance sampling oracle and achieves faster convergence than standard methods, with experiments showing up to 6× faster convergence to the same training loss.