arXiv AI

HOMURA: Taming the Sand-Glass for Time-Constrained LLM Translation via Reinforcement Learning

The paper introduces Sand-Glass, a benchmark for evaluating translation under syllable-level duration constraints, and proposes Homura, a reinforcement learning framework that optimizes the trade-off between semantic preservation and temporal compliance. Homura uses a constrained reinforcement learning objective with a dynamic syllable-ratio reward to effectively control output length. Experimental results show that Homura outperforms strong baselines, achieving precise length control while maintaining semantic adequacy.

arXiv AI
Sep 7

Reinforcement Learning for improving Large Language Models' Catalan text simplification capabilities

The paper explores using reinforcement learning to enhance automatic text simplification for low‑resource languages, focusing on Catalan. It introduces a new reward function that blends the SARI metric with penalty terms, and applies Group Relative Policy Optimization (GRPO) to fine‑tune the IberianLLM‑7B‑Instruct model on the ASSET dataset. Post‑training, the model shows improved simplification performance on two Catalan benchmarks and reduces prior negative behaviors, though cross‑lingual transfer from English, Spanish, and Catalan translations of ASSET does not yield significant gains on an out‑of‑domain benchmark.

By Arnau Ayguad\'e Domingo, Stefan Bott, Horacio Saggion
arXiv AI
Jun 9

Rewrite to Translate, Translate to Reward: Reinforcement Learning for Source Rewriting in Machine Translation

arXiv:2606. 08011v1 Announce Type: cross Abstract: Although directly prompting off-the-shelf Large Language Models (LLMs) to generate meaning-preserving source rewrites can effectively enhance Machine Translation (MT) quality, doing so requires manually tuning prompts for different MT models.

By Boxuan Lyu, Haiyue Song, Zhi Qu, Hidetaka Kamigaito, Kotaro Funakoshi, Manabu Okumura
arXiv Machine Learning
Jun 2

HMPO: Hybrid Median-length Policy Optimization for Chain-of-Thought Compression

arXiv:2606. 01934v1 Announce Type: new Abstract: Large language models achieve remarkable performance via extended chain-of-thought (CoT) reasoning, yet this lengthy process incurs substantial inference overhead.

By Minghui Zheng, Hongxu Chen, Huimin Ren, Hongsheng Xin, Xiaoyang Qu, Ze Wang, Shuling Yang, Ziyu Peng, Kaike Zhang, Pan Zhou, Kun Zhan
arXiv AI
Sep 2

Skill Reuse as Compression in Agentic RL

The paper introduces ReuseRL, a method that applies the Minimum Description Length principle to agentic reinforcement learning. By extracting a shared skill dictionary from successful trajectories and adding a segmentation cost to the RL objective, ReuseRL discourages idiosyncratic behaviors and promotes reusable abstract patterns. Experiments on ALFWorld, TextWorld-Cooking, and Countdown-Stepwise show that ReuseRL improves both in‑distribution and out‑of‑distribution success compared to vanilla GRPO and other baselines.

By Zhikun Xu, Yu Feng, Jacob Dineen, Taiwei Shi, Jieyu Zhao, Ben Zhou
arXiv AI
Jul 7

Evolutionary Guided Decoding: Iterative Value Refinement for LLMs

arXiv:2503. 02368v4 Announce Type: replace-cross Abstract: While guided decoding, especially value-guided methods, has emerged as a cost-effective alternative for controlling language model outputs without re-training models, its effectiveness is limited by the accuracy of the value function.

By Zhenhua Liu, Lijun Li, Ruizhe Chen, Yuxian Jiang, Tong Zhu, Zhaochen Su, Wenliang Chen, Jing Shao
Hugging Face Trending Papers
Jul 20

Token-Level Off-Policy Learning for Faithful Generation Under Distribution Shift

We propose Token-Level Off-Policy Labeling (TOPL), an off-policy training paradigm that reframes post-training as a token-level correctness prediction task. Our key intuition is that by training the model to distinguish good and bad tokens in a response, we naturally guide the model towards generating good tokens, while avoiding the pitfalls that come with directly training the model to generate off-policy tokens.