Computer vision

Detection, segmentation, depth and recognition research, plus the vision backbones that keep displacing the last generation.

3,429 stories · RSS feed

arXiv Computation and Language
Aug 27

ClueWeaver: Reward-Guided Dual-Agent Evidence Reasoning for Compact LLMs on Literary Long Narratives

ClueWeaver is a dual-agent framework designed to enable compact, locally deployable language models to answer questions about long literary narratives. The Finder agent retrieves passages that contain answer-critical clues, while the Interpreter agent derives the answer from those passages, generates rationales with paragraph-ID citations, and performs self-calibration for high-risk questions. Both agents are trained with reward-guided reinforcement learning to prioritize evidence retention, correctness, grounding, and concise explanations, resulting in improved performance and inspectability over end-to-end prompting.

By Jihao Zhu, Zhiwei Yang, Wenxiao Zhang, Junqian Zhao, Qi You, Fangqi Wang, Zheyuan Deng, Hanzhe Yang, Yu Liu, Jin B. Hong
arXiv Machine Learning
Aug 27

Robust CurveMoE: Multi-Norm Adversarial Defense for Mixture-of-Experts Models via Mode Connectivity

Robust CurveMoE is a mixture‑of‑experts framework that protects neural networks against perturbations defined by multiple norm constraints. It connects norm‑specialized models through a low‑loss path, selectively expertises only influential layers, and shares the rest of the parameters across routing paths. The method introduces contribution‑guided partial updating to reduce curve‑construction cost and provides a theoretical bound on the objective gap between partial and full optimization, achieving consistent improvements in clean, norm‑specific, and Union accuracy on CIFAR‑100 and ImageNet‑100.

By Xu Zhang, Ren Wang
arXiv Machine Learning
Aug 27

Learning from waste: Machine Learning for health risk prediction and computer vision-based sorting in Ghana

The study applies machine learning to address Ghana’s solid waste disposal challenges, using a Random Forest classifier to predict illness categories from waste practices and demographics, achieving a macro F1 score of 0.63. A MobileNetV2 image classifier was also developed for automated waste sorting, reaching 88.2% accuracy and a macro F1 of 0.87 on 415 images. These quantitative results confirm a previously qualitative link between waste disposal and health, while demonstrating the feasibility of low‑cost, camera‑based sorting in resource‑constrained settings.

By Hilda Adwubi Osei, Catherine Tenewaa Osei, Desdemona Yaa Asobayire
arXiv Computer Vision
Aug 27

Bootstrapping a 4D LiDAR Annotation Tool from Video Foundation Models

The paper introduces LiDAR‑SAM2, a framework that converts the 2D video foundation model SAM2 into a scalable source of supervision for 4D LiDAR data. By projecting SAM2 video masks into multi‑view LiDAR space and aggregating them temporally, the method automatically generates temporally coherent LiDAR labels without human annotation. Experiments on SemanticKITTI show that these automatically produced semantic and panoptic labels achieve quality close to full human annotation, enabling models trained on them to approach the performance of fully supervised systems.

By Jihun Kim, Hyun-Kurl Jang, Hyemin Yang, Jinnyeong Yang, Hyeokjun Kweon, Kuk-Jin Yoon
arXiv Computer Vision
Aug 27

Semantics-Aware Hierarchical Consensus Learning for Remote Sensing Image Classification

The paper introduces Semantics-Aware Hierarchical Consensus (SAHC), a deep learning framework that incorporates hierarchical-level-specific classification heads and cross-level probability projectors to leverage predefined label hierarchies in remote sensing image classification. SAHC fuses direct and projected predictions into a geometric consensus distribution, enabling self-consistent training and optional hierarchy-aware inference. Experiments on two benchmark datasets demonstrate the method’s effectiveness in guiding network learning and its robustness across varying spectral and spatial resolutions.

By Giulio Weikmann, Gianmarco Perantoni, Lorenzo Bruzzone
arXiv Computer Vision
Aug 27

SPARK-SAM: Learning How to Prompt and Respond for Infrared Small Target Segmentation

SPARK‑SAM is a new approach that adapts the Segment‑Anything Model (SAM) for infrared small‑target segmentation by learning target‑domain response knowledge and conditioning the decoder with an image‑conditioned joint self‑prompt state. In experiments on three IRSTD benchmarks, SPARK‑SAM achieves IoU scores of 75.78%, 86.49%, and 68.34% with only 0.726 M additional parameters, outperforming 14 retrained SAM variants. The method combines benchmark‑mask supervision with reliability‑aware response guidance, and ablations show consistent accuracy gains from response guidance and high‑resolution prompt refinement.

By Aji Mao, Zhenming Peng, Bailin Mu, Tian Pu
arXiv Computer Vision
Aug 27

Steer the Sampling, Not the Kernel Grid: Geometry-Guided Sampling Operator for Volumetric Segmentation

The paper introduces a geometry‑guided sampling operator that directs feature sampling rather than altering convolution kernels in 3D encoder‑decoder networks. By predicting local orientations and bounded step sizes, the operator samples symmetrically around each voxel, generating compact geometric and boundary cues that improve fine‑structure segmentation. Replacing stride‑1 and stride‑2 operations in a 3D U‑Net yields consistent gains on BraTS, MSD Hepatic Vessel, and TDSC‑ABUS datasets, with better boundary metrics and fewer parameters, and the operator can be integrated into other backbones without architectural changes.

By Sizhe Wang, Himashi Peiris, Zhaolin Chen
arXiv Computer Vision
Aug 27

Saliency-Depth Conditioning for Zero-Shot Segmentation of Communication-Tower Components in Cluttered UAV Imagery

The paper introduces a saliency-depth conditioning approach for zero‑shot segmentation of communication‑tower components in cluttered UAV imagery. By combining appearance‑based saliency with monocular relative depth, the method creates a coarse tower prior that suppresses irrelevant background, and integrates this module with Grounded‑SAM and SAM 3 to produce SD‑Grounded‑SAM and SD‑SAM 3. Experiments on the TOW‑300 dataset show that SD‑SAM 3 achieves the best instance‑segmentation performance while SD‑Grounded‑SAM reduces false positives, with ablations confirming the complementary benefits of saliency, depth, and box refinement.

By Ali Lesani, Chul Min Yeum, Su-Min Kang
arXiv Computer Vision
Aug 27

VisDocAgentBench: Benchmarking Agents for Visually Rich Document Retrieval

VisDocAgentBench is a closed‑corpus benchmark that evaluates static versus agentic retrieval for visually rich documents, using 2,375 pages from 100 documents and 120 queries that span direct, one‑bridge, and two‑bridge evidence structures. The benchmark includes semantic, relational, and visual queries, full‑document review, and hard‑negative validation. Results show that a strong visual retriever performs well on direct items but poorly on two‑bridge items, while agents improve performance, especially when using visual retrieval and iterative search capabilities.

By Lexiang Hu, Yanzhao Zhang, Mingxin Li, Dingkun Long, Yikang Li, Fuwei Zhang, Yisen Wang, Zhouchen Lin
arXiv Computation and Language
Aug 27

RefLAM: A Reference-Grounded Line Annotation Pipeline for Historical Arabic Manuscripts

RefLAM is a pipeline that converts manuscript page images and clean transcriptions into validated, line-level ground truth for Arabic handwritten text recognition. It combines a deep‑learning page‑segmentation model, a multimodal large language model for structured OCR, and a diacritic‑agnostic fuzzy alignment engine that assigns a confidence score to each line, with a provable correctness guarantee for perfect scores. Using RefLAM, the authors achieved a 75× speedup over manual annotation and released AraMS‑28k, a dataset of 14 historical Arabic manuscripts with detailed annotations.

By Mohamed Guechaoui, Mohamed Diaa Zellagui, Souleyman Chaib, Sahraoui Dhelim
arXiv Machine Learning
Aug 27

Advancements in Content-Based Image Retrieval: A Comprehensive Survey of Relevance Feedback Techniques

This survey reviews content‑based image retrieval (CBIR) systems, highlighting their use of visual content for image search and their importance in object detection. It discusses key challenges such as the semantic gap and scalability, and examines relevance feedback (RF) techniques—including long‑term and short‑term learning, weight optimization, and active learning—to iteratively refine search results. The paper also explores machine‑learning and deep‑learning approaches, particularly convolutional neural networks, to improve CBIR accuracy and relevance.

By Hamed Qazanfari, Mohammad M. AlyanNezhadi, Zohreh Nozari Khoshdaregi
arXiv Computer Vision
Aug 27

Improving Cross-Site Whole-Heart Segmentation

The paper presents a modality‑routed 3D cardiac segmentation pipeline that combines TotalSegmentator‑initialized nnU‑Netv2 models with site‑characterized, label‑preserving appearance augmentation. By analyzing measurable image properties across sites, the authors design a bias‑field plus Bezier augmentation strategy that smooths spatial intensity perturbations and remaps intensities nonlinearly, followed by class‑wise largest‑connected‑component cleanup. On held‑out validation splits, this approach raises CT mean Dice from 0.8350 to 0.9135 and MRI mean Dice from 0.7695 to 0.7830 while reducing HD95, demonstrating improved cross‑site robustness in limited‑data whole‑heart segmentation.

By Tanish Mudaliar, Justin Li, Daniel Lin, Julianna Vo, Kaitao Liao, Xin Wang, Shu Hu
arXiv Computer Vision
Aug 27

3DGAA: Realistic and Robust 3D Gaussian-based Adversarial Attack for Autonomous Driving

The paper presents 3DGAA, a fabrication-first framework that generates view-consistent, geometry-preserving adversarial wraps for vehicles using 3D Gaussian splatting optimization. It ensures consistency across viewpoints, illumination, and occlusion while limiting changes to vehicle geometry, producing realistic print-only textures that significantly reduce detection confidence and average precision in simulations and physical tests. Ablation and efficiency studies analyze the impact of physical filtering, augmentation, and shape-consistency regularization, and the method demonstrates robustness against common preprocessing defenses and cross-detector transferability.

By Yixun Zhang, Lizhi Wang, Junjun Zhao, Wending Zhao, Feng Zhou, Yonghao Dang, Jianqin Yin
arXiv Machine Learning
Aug 27

The Dialect Tax: Dialectal Biases Persist throughout the Language Modeling Pipeline

The study investigates why language models exhibit systematic performance gaps across English dialects, a phenomenon termed the "dialect tax." Using parallel dialect corpora that preserve meaning while altering surface form, the authors confirm that models treat Standard American English and dialectal texts as semantically equivalent, yet find representational disparities that persist through tokenization, pre‑training, post‑training, and inference. Even a character‑level tokenizer does not eliminate input/output asymmetries or accuracy gaps, and dialect pairs produce more divergent gradient updates than unrelated Standard texts, indicating that dialectal content is harder for models to learn.

By Elle
arXiv Computation and Language
Aug 27

Padamitra: Grounded Glossary Generation for Classical Sanskrit

The paper introduces grounded glossary generation, a structured NLP task that asks models to recover semantically meaningful Sanskrit phrases and provide translation‑grounded meanings from a sloka‑translation pair, mirroring the traditional patha commentary practice. A benchmark of 31,316 sloka‑translation‑glossary triples from the Valmiki Ramayana and Srimad Bhagavatam is built, evaluated with Jaccard for phrase recovery and Meaning Faithfulness for semantic consistency. Experiments with Gemma‑3n‑E4B, Gemma‑3‑12B, Phi‑4, and Qwen3.5‑9B show that instruction fine‑tuning outperforms prompting, and explicit segmentation further improves results, though over‑segmentation of sandhi and samasa compounds remains the main error source, highlighting morphological modeling as a key bottleneck.

By Manoj Balaji Jagadeeshan, Sai Pragnaan Marala, Pawan Goyal
arXiv Computer Vision
Aug 27

SEG-SAM: Semantic-Guided SAM for Unified Medical Image Segmentation

SEG-SAM is a unified medical image segmentation model that builds on the Segment Anything Model (SAM) by integrating semantic medical knowledge. It introduces a semantic‑aware decoder separate from SAM’s original decoder to handle both semantic segmentation of prompted objects and classification of unprompted objects. The model also incorporates key medical category characteristics from large language models via a text‑to‑vision semantic module and uses a cross‑mask spatial alignment strategy to improve overlap between predictions, achieving superior performance over existing SAM‑based and task‑specific methods.

By Shuangping Huang, Hao Liang, Qingfeng Wang, Chulong Zhong, Zijian Zhou, Miaojing Shi
arXiv Computer Vision
Aug 27

Forge4D: Feed-Forward 4D Human Reconstruction and Interpolation from Uncalibrated Sparse-view Videos

Forge4D is a feed‑forward model that reconstructs temporally aligned 4D human representations from uncalibrated sparse‑view videos, enabling both novel view and novel time synthesis. It achieves this by jointly streaming 3D Gaussian reconstruction with dense motion prediction, using learnable state tokens for temporal consistency and a self‑supervised retargeting loss for motion prediction. Extensive experiments confirm its effectiveness on in‑domain and out‑of‑domain datasets.

By Yingdong Hu, Yisheng He, Jinnan Chen, Weihao Yuan, Kejie Qiu, Zehong Lin, Siyu Zhu, Zilong Dong, Steven Hoi, Jun Zhang
arXiv Computer Vision
Aug 27

Synergistic Modality-and-Slice Memory Framework for Cross-Modal 3D Brain Tumor Segmentation

The paper introduces MSM‑Seg, a dual‑memory segmentation framework for 3D multi‑modal brain tumor segmentation. It combines a modality‑and‑slice memory attention module to capture cross‑modal and spatial‑slice dependencies, a multi‑scale category‑agnostic prompt encoder for whole‑tumor guidance, and a modality‑adaptive fusion decoder to integrate complementary decoding information. Experiments on various MRI datasets show that MSM‑Seg surpasses state‑of‑the‑art methods for metastases and glioma tumor segmentation.

By Yuxiang Luo, Qing Xu, Hai Huang, Yuqi Ouyang, Xiangjian He, Zhen Chen, Wenting Duan, Jiebo Luo
arXiv Computer Vision
Aug 27

UltraPIPS: Improving model perception in B-mode ultrasound with foundation models

UltraPIPS introduces domain‑specific foundation models for measuring perceptual similarity in B‑mode ultrasound images. The study shows that ultrasound‑trained LPIPS backbones better correlate with downstream tasks such as classification, segmentation, and reconstruction than natural‑image or general medical models. Optimizing LPIPS loss with an ultrasound backbone yields a strong balance between reconstruction quality and realism, and the authors provide an open‑source library for these metrics.

By Tal Grutman, Tali Ilovitsh