arXiv Machine Learning
Towards Scalable RLVR: Multimodal Instruction Following Data Synthesis and Distillation
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv AI
Jun 30
Beyond SFT-to-RL: Pre-alignment via Black-Box On-Policy Distillation for Multimodal RL
arXiv:2604. 28123v3 Announce Type: replace-cross Abstract: The standard post-training recipe for large multimodal models (LMMs) applies supervised fine-tuning (SFT) on curated demonstrations followed by reinforcement learning with verifiable rewards (RLVR).
arXiv Computation and Language
Aug 27
VISA: Agentic Self-Evolving Data Synthesis for Multimodal Instruction Following
VISA (Visual Instruction Synthesis Agent) is an agentic framework that transforms multimodal instruction synthesis into a self‑evolving loop. Each cycle analyzes images to filter constraints, samples new constraint sets, generates candidate instructions, and verifies them using executable tools and large language model judges. Failed samples trigger diagnostic recovery, while accepted samples are evaluated against the target model to estimate difficulty, with all feedback written back to memory to adapt future rounds and provide reward signals for reinforcement learning.
arXiv AI
Jun 8
Teaching the Way, Not the Answer: Privileged Tutoring Distillation for Multimodal Policy Optimization
arXiv:2606. 07000v1 Announce Type: new Abstract: Recent post-training methods, particularly Reinforcement Learning with Verifiable Rewards (RLVR), have significantly enhanced the reasoning ability of Large Vision-Language Models (LVLMs).
arXiv AI
Sep 25
Pistis Technical Report
The Pistis Technical Report introduces the Pistis model family, comprising 27B- and 9B-parameter multimodal large language models built on Qwen3.6 and Qwen3.5. The models are developed through a scalable post‑training framework that begins with large‑scale multimodal supervised fine‑tuning and then applies Interleaved Distillation and Reinforcement Learning (IDRL) to integrate on‑policy distillation and reinforcement learning within a single training loop. Two specialized variants—Pistis‑Thinking for deep multimodal reasoning and Pistis‑Agentic for long‑horizon planning, iterative reasoning, and tool use—are produced at both scales, and a system‑level method called Pistis‑Auto‑Harnessing (PAH) further improves inference harness performance without updating model parameters.
arXiv Computation and Language
Oct 9
MiMo-V2.6: Scaling Reinforcement Learning Towards Self-Improvement
MiMo‑V2.6 is an omni‑modal reinforcement learning framework that scales compute to advance large foundation models toward self‑improvement. It trains on a broad multimodal corpus before RL, then scales RL along three axes: larger batches and throughput, diverse environments across code, general, visual, and cyber domains, and enhanced grader compute for better reward signals. The system incorporates infrastructure such as a frozen MoE router, multi‑layer defense against reward hacking, unified trajectory representation, high‑concurrency rollout, and training‑inference consistency, and it is open‑source to support reproducibility and further research.
arXiv AI
Oct 1
Reinforcing Multimodal Reasoning via Token-Level Perception-Grounded Advantage Estimation
arXiv:2609.39168v1 Announce Type: new Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) has improved the reasoning capabilities of Multimodal Large Language Models (MLLMs), yet existing...