LangStreet introduces a persistent language field for anchor‑decoded street Gaussian scenes, addressing the breakdown of language Gaussian fields when geometry and appearance vary with camera views. The method relies on semantic ownership, where persistent decoder slots and parent anchors own the language field while transient children route observations. By accumulating additive directional evidence at slots and completing weakly supported slots with anchor‑aligned evidence, LangStreet achieves near‑maximal performance with significantly reduced storage, as demonstrated on KITTI, Virtual KITTI, and Waymo datasets.
arXiv:2609.15137v1 Announce Type: cross Abstract: 3D Gaussian language fields provide an explicit, spatially grounded representation for 3D visual question answering (VQA), but their dense semantic f...
The paper introduces Linguistic Trajectory Encoding (LTE), a hybrid representation that compresses dynamic object motion histories using natural language descriptions, sparse spatial anchors, and visual anchors. LTE adapts compression to motion complexity by anchoring periods without reliable observations to the last seen location while preserving accuracy with geometric waypoints and linguistic descriptions. Evaluated on the newly constructed Spatial Memory Benchmark (SMB) from EgoLife multi‑day recordings, LTE achieves 45.3 % success in semantic trajectory retrieval and 48.7 % in long‑horizon object retrieval, outperforming prior structured‑memory and VLM baselines, and compresses trajectories 8.7×–26.1× with sub‑second query latency on 24‑hour video.
The paper introduces a training‑free, target‑aware refinement method for 3D Gaussian language maps distilled from vision‑language features. By blending the visual embedding within user‑specified target regions with the corresponding text embedding using a convex combination, the method improves target localization while preserving open‑vocabulary queryability. The authors derive the optimal blending coefficient analytically, avoid hyperparameters, and demonstrate consistent gains across three benchmarks, including a real‑world robot navigation demo.
Feed-forward 3D Gaussian Splatting enables efficient novel-view synthesis without per-scene optimization, but most existing methods assume a fixed set of context views and process them jointly. This limits their applicability to online scenarios where calibrated views arrive sequentially and the scene must be updated causally.
SpatialMem is a memory-based system that fuses 3D geometry, semantics, and language into a single queryable representation. Starting from casually recorded egocentric RGB video, it reconstructs metrically scaled interior scenes, identifies structural 3D anchors (walls, doors, windows), and links open‑vocabulary object nodes to evidence patches, visual embeddings, and two‑layer descriptions at 3D coordinates, enabling spatial‑relation reasoning, language‑directed retrieval, and QA without special sensors. The system is evaluated at scale on public benchmarks, achieving competitive scores on VSI‑Bench, OpenEQA, and ScanNet, while maintaining a compact index of about 14 KB per scene.
V-LUMEN introduces a visual lookup memory for vision‑language models by discretizing visual features with a frozen codebook, creating spatially aware memory keys via hashing, and modulating them with text‑conditioned relevance signals. The retrieved embeddings are fused into a frozen Qwen3‑VL‑2B backbone through a gated residual pathway, yielding substantial improvements on five reasoning‑intensive benchmarks and zero‑shot transfer to three disjoint image datasets. At a matched parameter budget, reallocating capacity from LoRA or MoE adapters to V‑LUMEN preserves accuracy on two backbones, demonstrating that external visual memory offers an independent capacity axis. whyItMatters":"The paper shows that adding a non‑parametric visual lookup memory can significantly boost vision‑language model performance without increasing per‑token compute, offering a new way to scale such models."
CoRef-GS introduces a cooperative referring Gaussian splatting framework for multi‑agent scene understanding, enabling robots to ground object‑ and relation‑centric language queries across independently reconstructed maps. The method builds local open‑vocabulary instance‑aware Gaussian maps, aligns them using a cross‑agent module that enforces geometric and semantic consistency, and grounds queries with a view‑conditioned mask relation graph. Experiments on a new dual‑quadruped benchmark show significant improvements, reducing rotation error from 2.58° to 0.15° and raising real‑world referring mIoU from 52.6% to 68.8%.
GS‑VLA introduces a lightweight, plug‑and‑play framework that uses a 4 M‑parameter 3D‑Gaussian canonicalizer to adapt frozen Vision‑Language‑Action (VLA) policies to viewpoint shifts without retraining the policy. By treating viewpoint changes as a localized novel‑view synthesis problem under a locality assumption, the method normalizes observations through a scene‑ and policy‑independent disocclusion task. Experiments on the LIBERO benchmark demonstrate that GS‑VLA recovers a large portion of performance lost due to camera displacement, improving results across different policy architectures, unseen task suites, and perturbation scales. whyItMatters":"The approach offers a computationally efficient alternative to costly fine‑tuning or generative augmentation, enabling robust VLA deployment in real‑world settings where camera configurations may vary."
arXiv:2607. 10578v1 Announce Type: new Abstract: Existing hypotheses represent a concept in an LLM as a single point, a linear direction, or a Gaussian cluster, yet it remains unclear how and why such structures emerge.
arXiv:2608.29924v1 Announce Type: cross Abstract: Large Vision-Language Models (LVLMs) are prone to hallucinations: they fluently describe objects, attributes, and scenes that are not in the image. W...
CatchUp is a training‑free framework that improves efficiency in large vision‑language models by depth‑aware re‑admission of visual tokens. It repeatedly evaluates query‑conditioned relevance, processes only selected tokens per decoder interval, and preserves deferred tokens for later re‑admission, using reference‑to‑current transition signals to approximate missed representation evolution. Experiments on three referring‑expression grounding datasets and six multimodal benchmarks show that CatchUp retains over 92% of full‑model performance with 88.9% fewer active tokens and reduces end‑to‑end latency by 8.0% at a 64‑token budget.
MemoryVLN introduces a memory-driven navigation system that compresses long navigation histories and reuses past spatial experience. It uses forgetting-curve-inspired temporal sampling for long sequence compression, an ego-aware memory representation that fuses 2D semantic, 3D geometric, and 6D pose tokens, and a hierarchical memory bank for in-episode and context-level memory management. The system retrieves top‑K relevant memories to augment current visual tokens, achieving significant improvements on R2R‑CE and RxR‑CE benchmarks.