Hugging Face Trending Papers
Turn-level Multiscale Density Ratio Estimation for LLM Agents
Read the original on Hugging Face Trending Papers →The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.
Related stories
arXiv AI
Sep 16
Turn-level Multiscale Density Ratio Estimation for LLM Agents
The paper introduces Turn-level Multiscale Density Ratio Estimation (tlm‑DRE), a post‑training technique for large language model agents that assigns varying weights to different turns and employs asymmetric token‑level training based on positive‑negative gaps across multiple turns. Unlike conventional alignment methods such as PPO, DPO, DIL, and GRPO, which mainly target single‑turn tasks, tlm‑DRE is designed for complex multi‑turn reasoning. Experiments on diverse agent benchmarks demonstrate that tlm‑DRE competes with traditional alignment methods and enables LLMs to perform robustly in both in‑domain and out‑of‑domain multi‑turn tasks.
arXiv AI
Aug 19
PlanPO: Group Planning-Aware Policy Optimization for Multi-Turn Agentic LLMs
PlanPO introduces a group planning-aware policy optimization method for multi-turn agentic large language models, addressing the issue of advantage collapse caused by treating all successful trajectories equally. By incorporating coarse-to-fine advantage signals that reflect differences in trajectory and turn lengths, PlanPO encourages agents to learn generalizable planning and generation behaviors. Experiments show a 27.2% average improvement over GRPO on benchmarks such as ALFWorld, WebShop, and SciWorld, with minimal extra training cost.
arXiv AI
Aug 25
From Isolation to Alignment: Unified LoRA for Efficient Multi-Task Learning
The paper introduces Align‑LoRA, a unified LoRA framework for multi‑task learning that replaces complex, isolated adapter designs with a single‑adapter model enhanced by a higher rank and an explicit alignment loss. It demonstrates that a router‑free, multi‑head model with high inter‑head redundancy can outperform more elaborate baselines, and that a unified LoRA can achieve competitive performance while enabling weight merging and zero inference latency. Extensive experiments and theoretical analysis confirm that Align‑LoRA surpasses prevailing approaches, offering a simpler, production‑friendly paradigm for parameter‑efficient fine‑tuning of large language models.
Hugging Face Trending Papers
Jul 6
Turning Off-Policy Tokens On-Policy: A Plug-in Approach for Improving LLM Alignment
Reinforcement learning (RL) post-training for large language models (LLMs) follows a efficient paradigm of "rollout then update", which inevitably results in off-policy training data. To resolve this, Importance sampling (IS) is proposed, while the token-level ratios compound over long sequences, causing severe variance exploded.
arXiv AI
Oct 9
Predicting Alignment Generalization with Value Representations
The paper introduces the task of predicting alignment generalization, which measures how fine‑tuning a language model to follow a specific value affects its behavior across a broad set of other values. Through a large‑scale study of 66 values, the authors show that representations derived from model activations when applying values in context outperform textual description‑based methods, achieving a correlation of 0.45 versus 0.05. They further demonstrate that these representations can predict downstream robustness and help construct a taxonomy of LLM values based on empirical generalization dynamics.
arXiv AI
1d ago
A Comprehensive Framework for All Stages of Large-Model Training: From Data and Tokenization, through Pretraining, to Post-Training Alignment
arXiv:2610.33451v1 Announce Type: new Abstract: The capabilities of large language models (LLMs) and multimodal LLMs (MLLMs) arise from the synergy of the full training pipeline, including data and t...