← Back to all news
arXiv Machine Learning

Mixture-of-Experts for Cryptocurrency Order Execution: Training Stability, Tail Risk, and Failure Modes

The study investigates whether mixture‑of‑experts (MoE) architectures improve deep reinforcement‑learning policies for cryptocurrency order execution. Using 5‑minute BTC/USDT limit‑order‑book data, the authors compare vanilla Double Deep Q‑Learning (DDQL) with K‑means‑partitioned MoE experts (K=2,4,8) and dense networks matched in size to the expert budgets. Across 100 training seeds, none of the MoE variants reduce mean implementation shortfall relative to DDQL, and all perform worse than simple benchmarks (TWAP and immediate liquidation). The authors find that training‑specification issues—particularly exploration schedules—drive most failures, while MoE partitioning offers no intrinsic benefit and can mask underlying problems. They also report that higher K increases tail risk and that performance variability depends heavily on seed selection.

Related stories

arXiv AI
Oct 9

StoreBench: A Live-Commerce Environment for Evaluating and Training Autonomous Operator Agents

StoreBench is a live‑commerce environment where an autonomous agent runs a mid‑size online apparel store on a production‑grade backend, facing continuous customer orders, supplier price changes, and market shocks. The agent uses the same 29 merchant tools as a human operator, with a windowed operation budget that ties simulated time to actions, ensuring that model latency does not affect the simulation. Seven large language models were evaluated across 11 scenarios of 30–45 days and a full simulated year, but none matched the scripted smart‑triage policy, and human experts outperformed all models.

By Daksh Raghuvanshi, Ved Vedere, Yifan Wang
llmsagentsreinforcement-learningbenchmarks
More like this →
arXiv Machine Learning
Oct 15

Selective Growing Q-Networks (SGQN): An Analysis of Adaptive Action Grids in Decoupled Q-Learning

Selective Growing Q‑Networks (SGQN) extend value‑based Q‑learning for continuous control by replacing uniform action‑grid growth with an adaptive grid that adds bins near high‑Q regions and prunes low‑Q bins, while maintaining decoupled Q‑learning and epsilon‑greedy action selection. On eight DeepMind Control Suite tasks, SGQN achieves the highest mean final return on four tasks, matches the best method within 1.1% on three more, and uses on average 7.5 of 9 available bins per dimension. Experiments show that the adaptive grid’s growth and pruning have limited impact on performance, whereas starting from a coarse grid and using epsilon‑greedy selection are critical for success.

By Parham Nooranbakht, Seyed Mahdi Basiri Azad, Joschka B\"odecker
reinforcement-learningefficiency
More like this →
arXiv AI
Jul 21

CADENCE: Closing the Reasoning Gap via Coverage-Adaptive On-Policy Distillation

arXiv:2607. 16955v1 Announce Type: cross Abstract: On-policy knowledge distillation transfers reasoning from large teachers to compact students, but existing approaches suffer three compounding failure modes: (i) cold-start collapse, where a fresh student assigns near-zero mass to teacher-preferred tokens; (ii) state-agnostic divergence scheduling, where time-only forward/reverse-KL interpolation ignores the student's coverage state; and (iii) binary reward sparsity, where pass/fail signals discard information from partially correct traces.

By Satyam Kumar, Saurabh Jha
efficiency
More like this →
arXiv AI
Sep 2

Explore More, Drift Less: Outcome-Only Reinforcement Learning Can Suffice for Long-Horizon Interactive Agents

The paper proposes CANOPY, a minimalist reinforcement learning protocol that addresses two common pitfalls—signal starvation and policy drift—in outcome‑only RL for long‑horizon interactive tasks. By scaling same‑task exploration, keeping updates on‑policy, and anchoring updates with KL divergence, CANOPY enables a Qwen3‑14B agent to achieve top leaderboard results on the AppWorld coding benchmark without auxiliary supervision or elaborate scaffolding. The approach also improves performance on SWE‑bench for a Qwen3.5‑9B model.

By Liming Pu, Xiaoxia Li, Yifu Liu, Teng Cao, Bin Yang
llmsagentsreinforcement-learningfine-tuningbenchmarks
More like this →
arXiv Machine Learning
Aug 6

Adaptive Finite-Budget Training for CVaR Risk-Aware Q-Learning

arXiv:2608. 04305v1 Announce Type: new Abstract: Risk-aware Q-learning (RaQL) provides a model-free, two-timescale estimator for dynamic risk objectives, but its finite-budget behavior remains fragile: fixed inner-loop hyperparameters can produce unstable value estimates, persistent Bellman residuals, and inefficient sample reuse.

By Yifan Wu, Junjie Lei, Wenjie Huang
reinforcement-learning
More like this →