arXiv AI
Jul 21
LookME: Lookup-Based Multimodal Embeddings for Layer Injection in Vision-Language Models
arXiv:2607. 16305v1 Announce Type: cross Abstract: Vision-Language Models (VLMs) have achieved strong progress in multimodal understanding.
Decoupled I/O-Dominant Pipelines for Large-Scale Whole-Slide Image Embedding Extraction
The paper introduces a decoupled, I/O-aware pipeline for extracting embeddings from large whole-slide images (WSIs). It splits the workflow into patch generation, parallel embedding inference, and sharded vector database ingestion, isolating data movement from compute to reduce I/O overhead. The resulting distributed vector database stores embeddings with rich metadata, enabling efficient filtering, retrieval, and downstream tasks such as classification and few-shot learning, especially in low-resource settings.
Related stories
arXiv Computation and Language
Sep 23
Efficient Iterative Retrieval with Heterogeneous Batching
Orthrus is a serving system that performs heterogeneous batching of embedding and generative models within a single inference loop. It uses chunked embedding with incremental pooling and workload‑aware batch composition to unify conflicting computational patterns. Experiments on four A100 GPUs show that Orthrus improves throughput by 1.28×–4.52× and reduces p99 latency by up to 55.8% compared to baseline deployments.
arXiv AI
Sep 10
FastE: Readout-Triggered Token Compression for LLM Embedding Inference
FastE is a training‑free, plug‑and‑play method that compresses token prefixes in large language model (LLM) embedding inference. It uses a shared fixed threshold on batch‑mean readout‑prefix alignment to decide when to compress and ranks prefix states by readout attention scores to keep the most important ones. Experiments on Qwen3‑Embedding models show that FastE can cut decoder‑backbone FLOPs by over 40% while preserving more than 99% of the original ranking quality across multiple benchmarks and tasks.
arXiv AI
Aug 11
Simple Token-Efficient Vision-Language Model for Case-level Pathology Synoptic Report Generation
arXiv:2605. 30716v2 Announce Type: replace-cross Abstract: Generating clinically useful pathology reports for pathology cases from whole-slide images (WSIs) is challenging due to gigapixel resolution, long visual-token sequences, and the complexity of case-level reasoning, where a single case may contain multiple WSIs with heterogeneous tissues and ambiguous findings.
Hugging Face Trending Papers
Jul 26
PathSelect: Sequential Token Selection for Whole Slide Pathology
Gigapixel Whole-Slide Images (WSIs) present a fundamental computational bottleneck for vision-language models (VLMs) due to extreme sequence lengths. Existing approaches predominantly rely on spatial sampling or training-free pruning, which risk diluting weak but informative signals, leading to the loss of critical diagnostic evidence due to the spatially diffuse nature of pathological cues.
arXiv AI
6d ago
VisionWeave: Weaving Elastic Visual Representations as a Native Capability of MLLMs
VisionWeave introduces elastic visual representation weaving, a native capability for multimodal large language models that learns where and at what granularity to encode visual information. The method combines a gated spatial pooler for coarse representations with a granularity router that allocates content‑adaptive token usage, trained end‑to‑end on large‑scale data. Experiments on Qwen3.5‑4B and Qwen3.8‑27B show that VisionWeave can save 43.0% of tokens while preserving 98.9% of performance across eight benchmarks, and delivers significant throughput gains and latency reductions when deployed on the SGLang serving engine.
Hugging Face Trending Papers
Aug 24
ProxyFormer: A Dual-Stream Proxy Architecture for Ultra-Long Context and High-Resolution Generation
The quadratic growth of attention computation and key-value (KV) cache with respect to sequence length is a central bottleneck for ultra-long-context language models and high-resolution generative mod...
arXiv Computer Vision
Sep 30
UniMedSeg: Unified In-Context Learning for Multi-Paradigm 2D/3D Medical Image Segmentation
arXiv:2607.12896v3 Announce Type: replace Abstract: Medical image segmentation foundation models are expected to generalize across diverse clinical scenarios, yet existing universal methods remain fr...
arXiv Machine Learning
1d ago
STAPLE: Static Sparse Upcycling of Pretrained Checkpoints via Embedding Modules
arXiv:2610.12599v1 Announce Type: new Abstract: Sparse computation can scale the parametric capacity of LLMs without proportionally increasing active computation, but pretraining sparse architectures...
arXiv AI
Jul 8
Analysis-by-Proxy: Localization Signals in VLMs Operating as Condition Encoders
arXiv:2607. 06445v1 Announce Type: cross Abstract: Vision-Language Models (VLMs) are increasingly utilized as the conditioning backbone for diffusion-based image editing due to their remarkable multimodal reasoning capabilities.
arXiv Machine Learning
Jul 31
Explorative Modeling: Unlocking a Third Pretraining Axis and End-to-End Generation
arXiv:2607. 27372v1 Announce Type: new Abstract: The deep learning revolution, kicked off by AlexNet, taught us that end-to-end training beats decomposing a problem into hand-designed stages.
arXiv AI
Sep 3
NeoMME: A Single-Tower Multimodal-Native Multilingual Foundation Encoder for Efficient Fine-Tuning and Inference
NeoMME is a family of 260M and 800M‑parameter multimodal‑native multilingual encoders that process text and raw image patches in a single bidirectional Transformer. Trained from scratch with a masked discrete‑diffusion objective conditioned on visible image patches, NeoMME supports a 16,384‑token context, enabling encoding of up to two 4K UHD images. In downstream tests, NeoMME‑Retriever models outperform all sub‑800M‑parameter baselines on the ViDoRe v3 benchmark and achieve twice the throughput of ColModernVBERT on an NVIDIA L40S, while hierarchical token pooling and asymmetric quantization compress embeddings 255× with minimal loss in retrieval performance.