arXiv Machine Learning August 6, 2026 By Boyao Wang, Zhihan Lei

SpecDrop: Parameter-Free Category-Conditioned Routing for Modular Specialization

Read the original on arXiv Machine Learning →

arXiv:2608. 04084v1 Announce Type: new Abstract: Mixture-of-experts (MoE) networks pursue specialization through learned routers, gates, and load-balancing losses, yet at matched total-parameter budgets learned routers can underperform equal-weight No-Routing baselines.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

arXiv AI

Jul 3

Do Newer Lightweight CNNs Perform Better Under Resource Constraints? A Controlled Multigenerational Study of Architecture, Initialization, Training Budget, and Efficiency

arXiv:2607. 01984v1 Announce Type: cross Abstract: Newer lightweight convolutional neural networks are often presented as improving predictive performance and deployment efficiency, but such claims require controlled evaluation.

By Tasnim Shahriar

arXiv AI

Jul 22

Where Should Optimizer State Live? Tiered State Allocation for Memory-Efficient Mixture-of-Experts Training

arXiv:2607. 19058v1 Announce Type: cross Abstract: Optimizer state is the largest single line item in the memory budget of mixture-of-experts (MoE) training: on a 6.

By Nuemaan Malik

llms benchmarks

arXiv Machine Learning

Jun 16

HyDRA: Hybrid Dynamic Routing Architecture for Heterogeneous LLM Pools

arXiv:2605. 17106v2 Announce Type: replace-cross Abstract: Production LLM deployments increasingly maintain heterogeneous model pools spanning order-of-magnitude cost differences.

By Aashna Garg, Siddharth Singha Roy, Jinu Jang, Federico Brancasi, Shengyu Fu

llms agents efficiency

arXiv AI

Aug 6

When does training on downscaled images yield the same gradients?

arXiv:2608. 04448v1 Announce Type: cross Abstract: Diffusion transformers deliver strong image generation, but their training cost grows superlinearly with resolution.

By Seunghyun Ji

llms diffusion fine-tuning

arXiv Machine Learning

Aug 11

FEAST: Federated Shared-Space Training for Resource-Heterogeneous Clients

arXiv:2608. 09250v1 Announce Type: new Abstract: Federated learning (FL) must serve devices with varying computational capabilities.

By Bostan Khan, Masoud Daneshtalab

arXiv AI

Aug 11

Router Sensitivity Under Lightweight Fine-Tuning Identifies Prunable Experts in Mixture-of-Experts Models

arXiv:2608. 07890v1 Announce Type: cross Abstract: Mixture-of-Experts (MoE) models decouple total parameters from per-token compute, but deployment still requires storing every expert.

By Ali Janati, Kaoutar El Maghraoui, Xinyi Luo, Wenyuan Shen, Owen Zou, Yankai Mao

fine-tuning efficiency benchmarks

Related stories

Do Newer Lightweight CNNs Perform Better Under Resource Constraints? A Controlled Multigenerational Study of Architecture, Initialization, Training Budget, and Efficiency

Where Should Optimizer State Live? Tiered State Allocation for Memory-Efficient Mixture-of-Experts Training

HyDRA: Hybrid Dynamic Routing Architecture for Heterogeneous LLM Pools

When does training on downscaled images yield the same gradients?

FEAST: Federated Shared-Space Training for Resource-Heterogeneous Clients

Router Sensitivity Under Lightweight Fine-Tuning Identifies Prunable Experts in Mixture-of-Experts Models