Existing scaling strategies for Multimodal Large Language Models (MLLMs) typically expand either model parameters or sequential inference computation, incurring substantial memory or latency overhead. More importantly, most existing methods fail to alter the rigid, fixed computation allocation between the Vision Transformer and the Large Language Model components, limiting task-specific optimization.
arXiv:2605. 25451v2 Announce Type: replace Abstract: Training multimodal large language models (MLLMs) is challenged by both model and data heterogeneity.
arXiv:2610.01640v1 Announce Type: cross Abstract: Vision-language models (VLMs) face a fixed-budget trade-off between processing more visual information for fine-grained perception and using a larger...
arXiv:2607. 22577v1 Announce Type: new Abstract: Scaling large language models (LLMs) has driven their success, yet dense Transformers couple capacity and computation: every parameter is activated for every token, making training and inference costs grow linearly with model size-a critical bottleneck as models approach trillion-parameter regimes.
The paper introduces the Capability-Driven Multimodal Scaling Law, a cross-family framework that predicts vision-language model (VLM) benchmark accuracy from a low-dimensional textual capability score extracted via PCA. By training over 150 VLMs on 34 large language models across seven families, the authors demonstrate that the law accurately extrapolates transfer rates from 8B to 72B‑parameter backbones, predicts full training trajectories, and generalizes to unseen model families. The study also reveals actionable insights, such as certain textual benchmarks negatively correlating with multimodal performance and base LLMs outperforming instruction-tuned counterparts as VLM backbones due to higher absorption rates.
Scaling Large Language Models (LLMs) has been driven mainly by enlarging the Transformer backbone, but for an already-strong model this requires another round of costly pretraining. We study whether an existing backbone can keep improving by allocating more computation to each token while leaving the Transformer backbone fixed.
arXiv:2606. 12688v1 Announce Type: cross Abstract: We are entering a new era of composite model architectures that integrate diverse components such as vision encoders, language backbones, diffusion and flow heads, audio codecs, action generators, and world-model predictors.
arXiv:2602. 14134v2 Announce Type: replace-cross Abstract: Multimodal Large Language Models (MLLMs) have demonstrated exceptional capabilities in high-level visual understanding.
EAServe introduces an encode-aware disaggregated serving framework for multimodal large language models (MLLMs), restructuring the traditional Prefill-Decode pipeline into a three-stage Encode-Prefill-Decode (EPD) system. By treating Encode as the control point, EAServe coordinates load‑adaptive micro‑batching, rate‑controlled offloading to prefill workers, and dynamic SM partitioning to balance GPU utilization across stages. Its Hybrid Auto Selection (HAS) layer optimizes GPU allocation, encode batch size, and offload ratio using capacity profiling and Bayesian optimization, achieving up to 4.3× higher goodput compared to NVIDIA Dynamo and 1.7× higher than vLLM on various MLLM architectures.
arXiv:2609.10355v1 Announce Type: cross Abstract: Video understanding has rapidly evolved toward video large language models (VideoLLMs): systems that couple video representations with pretrained lar...
arXiv:2602. 21788v2 Announce Type: replace-cross Abstract: Scaling long-context capabilities is crucial for Large Language Models (LLMs).
arXiv:2609.16722v1 Announce Type: new Abstract: Scaling Multimodal Large Language Models (MLLMs) to long-form video understanding is bottlenecked by the explosion of visual tokens, which saturates co...
The survey "From Models to Systems: A Comprehensive Survey of Efficient Multimodal Learning" reviews over 300 works on efficient multimodal learning (EML), proposing a structured taxonomy that spans model, algorithm, and system layers. It synthesizes how cross‑layer co‑design addresses the Efficiency‑Utility‑Privacy trade‑off and illustrates this through a case study of multimodal large language models. The paper also offers optimization blueprints for various domains, discusses a shift toward self‑regulating intelligence, and outlines open challenges for future EML research.