arXiv Machine Learning
Sep 23

Optimizing Denoising Trajectories in dLLMs: A Lightweight Evolutionary Heuristic Approach

The paper introduces a lightweight evolutionary heuristic scheduler for Diffusion Large Language Models (dLLMs) that optimizes denoising trajectories using CMA-ES. It addresses failure modes of existing confidence-based schedulers—EOS Overflow and Proximal Bias—by integrating multiple heuristic features with a contextual mean-field embedding, requiring only 393 trainable parameters. Evaluations on LLaDA and Dream across reasoning and planning benchmarks show consistent outperformance over strong baselines and recent state‑of‑the‑art methods.

By Zijian Zhao, Dian Jin, Xialiang Tong, Sen Li, Mingxuan Yuan
arXiv AI
Oct 19

Rethinking User Encoders: Routing Diffusion States for Scenario-Adaptive User Representation

The paper introduces FINDERS, a diffusion‑language‑model framework that replaces static user encoders with a dynamic routing system. FINDERS uses four operations—denoise, factorize, route, and compress—to create scenario‑adaptive user representations by organizing denoising states into transferable cross‑modal semantics and modality‑specific factors. The authors also build BehaviorQA, a large corpus with explicit behavioral rationales, and demonstrate through extensive experiments and online A/B tests that FINDERS outperforms existing methods across ten industrial scenarios.

By Junru Zhang, Jinyong Wen, Bin Dou, Wuliang Huang, Baokun Wang, Yu Cheng, Weiqiang Wang, Cheng Peng