Distributional Reward Calibration and Budget-Neutral Credit Assignment for Multimodal RLVR

The paper introduces TACE, a two‑stage estimator for advantage estimation in Group Relative Policy Optimization. Stage S1 replaces linear standardization with an optimal‑transport map to a standard normal, aligning reward distributions across tasks and limiting outliers. Stage S2 rank‑calibrates token‑level credit assignment, normalizing token weights to preserve total credit while redistributing it within trajectories. The authors prove theoretical guarantees—inter‑task moment equity, deterministic tail bounds, and mass‑conservation—that previous methods lack, and demonstrate that TACE achieves the best performance on most benchmarks in a multimodal reasoning suite.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

• multimodal
• benchmarks