arXiv Computer Vision
Learning to Orchestrate Vision Foundation Models for Multi-Task Dense Prediction
Read the original on arXiv Computer Vision →The Flow has not summarised this story yet — read it at arXiv Computer Vision.
Related stories
arXiv Computer Vision
4d ago
Task-State Adaptation with Prototype Memory for Multi-Task Dense Prediction
The paper introduces MemMTL, a multi‑task dense prediction framework that uses a compact task state derived from global visual context and refines it via a learnable prototype memory. This refined state informs task‑conditioned expert logits, which are combined with token‑level logits and routed through a sparse top‑k selection over a shared local expert bank. A task‑agnostic residual bank offers a common adaptation path, and both paths are added to the backbone feature before task‑specific prediction. The authors outline an evaluation protocol on NYUD‑v2 and PASCAL‑Context using SAM 3 and ViT‑L backbones to assess predictive quality, computational cost, and the contributions of task‑state conditioning, prototype retrieval, and sparse routing.
arXiv AI
Jun 3
PRISM: Synergizing Vision Foundation Models via Self-organized Expert Specialization
arXiv:2606. 03444v1 Announce Type: cross Abstract: Unifying the complementary strengths of diverse Vision Foundation Models (VFMs) into a single efficient model is highly desirable but challenged by the negative transfer inherent in monolithic distillation.
arXiv Machine Learning
Jul 31
S-GRPO: Unified Post-Training for Large Vision-Language Models
arXiv:2604. 16557v2 Announce Type: replace Abstract: Current post-training methodologies for adapting Large Vision-Language Models (LVLMs) generally fall into two paradigms: Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL).
arXiv AI
Jun 2
On the Limits of Token Reduction for Efficient Unified Vision Language Training
arXiv:2606. 01503v1 Announce Type: cross Abstract: Unified vision-language models (VLMs) integrate visual understanding and visual generation within a single autoregressive backbone, but their joint training is computationally expensive and largely overlooked from an efficiency perspective.
arXiv AI
Jun 15
Learning What to Predict: Downstream-Guided Task Design for Continued Pretraining
arXiv:2601. 22108v2 Announce Type: replace-cross Abstract: Continued pretraining is optimized with fixed self-supervised tasks but selected by downstream performance, creating a coarse feedback loop in which practitioners evaluate checkpoints, change data mixtures or objectives, and restart runs, while individual updates remain blind to target capabilities.
arXiv AI
Aug 5
Video-DeepResearch: Towards the Next-Generation Multimodal Deepresearch Agent
arXiv:2608. 03979v1 Announce Type: cross Abstract: We introduce Video-DeepResearch (Video-DR), extending multimodal agents from static images to continuous video streams, a setting that demands dense spatiotemporal grounding coupled with open-web exploration.