Computer vision

Detection, segmentation, depth and recognition research, plus the vision backbones that keep displacing the last generation.

3,426 stories · RSS feed

arXiv Computer Vision
Sep 7

Compositional Reward Models for Conditional Medical Image Generation

The paper introduces PRISM, a Compositional Reward Model framework that decomposes image quality into multiple verifier‑grounded stages for conditional medical image generation. By assigning distinct rewards for fine‑to‑coarse properties—such as intensity, texture, structural alignment, and semantic fidelity—and combining them via a Hierarchical Constrained Propagation mechanism, PRISM addresses shortcomings of single‑scalar reward approaches. Experiments on PanNuke, CeDeM, and ISIC datasets show that data generated with PRISM improves downstream model performance, achieving higher mDice, lower MRE, and increased F1 scores compared to baseline methods.

By Aayush Kumar Tyagi, Prathosh A. P., Mausam
arXiv AI
Sep 7

Sound-based Multi-Person 3D Pose Estimation

The paper introduces SoundMHPE, an encoder‑decoder framework that estimates 3D poses of multiple people using only acoustic signals. It addresses challenges such as overlapping acoustic signatures and inter‑person reflections by employing a multi‑scale acoustic encoder and a temporal pose decoder with attention. The authors created the 6‑hour Acoustic Multi‑person Pose (AMP) dataset and show that SoundMHPE outperforms baseline models.

By Yusuke Oumi, Yuto Shibata, Go Irie, Akisato Kimura, Yoshimitsu Aoki, Mariko Isogawa
arXiv Computation and Language
Sep 7

Retinal OCTA Phenotyping with LLM Reporting for Alzheimer's Disease

The study introduces an explainable OCTA pipeline for Alzheimer’s disease phenotyping that combines vessel segmentation, layer‑specific biomarker extraction, and label‑free phenotyping with large language model (LLM) reporting. Using 117 images from 39 subjects, the segmentation models achieved high ROC‑AUC (0.916–0.970) and Dice scores (0.695–0.781), and six vascular biomarkers were used to create subject‑level profiles for exploratory clustering. LLMs (GPT, Gemini, Llama) produced measurement‑grounded reports evaluated for citation faithfulness and diagnostic caution, offering a transparent, non‑diagnostic link between retinal vascular data and Alzheimer’s research.

By Progga Paromita Dutta, Jeba Maliha, Md Rafiul Kabir
arXiv Computer Vision
Sep 7

An Attention-Guided Global and Local Fusion Framework for Lesion-Focused Image Classification

The paper introduces an attention‑guided fusion framework that combines global and lesion‑focused local information for image classification. Using a three‑branch architecture built on DenseNet‑121, the model generates attention maps with Grad‑CAM, refines local features with CBAM, and adaptively fuses the two representations. Experiments on synthetic and real datasets, including skin, guava leaf, and grape leaf images, show that the fusion branch outperforms individual branches, achieving up to 97.75% accuracy on skin lesions and 99.64% on guava leaves.

By Mst Shafia Tasnima, Md Samaun Elaheea, Tanjim Taharat Aurpab, Md Musfique Anwar
arXiv AI
Sep 7

Commonsense Reasoning in Computer Vision: Foundations, Recent Advancements, and Future Directions

The paper surveys how commonsense reasoning is being integrated into computer vision, moving beyond traditional CNNs that only detect objects. It reviews methods that use knowledge graphs, scene graphs, neuro-symbolic models, and transformers to add contextual understanding, thereby improving object recognition and spatial reasoning. The authors also discuss current limitations such as dataset bias and knowledge gaps, and propose future research directions in cross‑modal reasoning, scalable knowledge injection, and hybrid architectures.

By Bahar Uddin Mahmud, Sumit Barua, Guan Yue Hong, Ajay Gupta, Hexu Liu
arXiv Computer Vision
Sep 7

WeakMCN: Multi-task Collaborative Network for Weakly Supervised Referring Expression Comprehension and Segmentation

WeakMCN introduces a multi-task collaborative network that jointly learns weakly supervised referring expression comprehension (WREC) and segmentation (WRES) using a dual-branch architecture. The WREC branch employs anchor-based contrastive learning and serves as a teacher for the WRES branch, while two novel modules—Dynamic Visual Feature Enhancement (DVFE) and Collaborative Consistency Module (CCM)—facilitate cross-task collaboration. Experiments on RefCOCO, RefCOCO+, and RefCOCOg show significant performance gains over single-task baselines, with up to 3.91% and 13.11% improvements on WREC and WRES respectively, and strong generalization in semi-supervised settings.

By Silin Cheng, Yang Liu, Xinwei He, Sebastien Ourselin, Lei Tan, Gen Luo
arXiv Computer Vision
Sep 7

Object Concepts Emerge from Motion

The paper introduces a biologically inspired framework that learns object‑centric visual representations from raw videos without human annotations or camera calibration. By using motion boundaries detected via optical flow and clustering to create pseudo‑instance masks, the method supervises a single‑image encoder with pixel‑level pairwise metric learning. Training on 195 million pseudo‑labeled frames and expanding to 421 million frames through Motion‑Verified Self‑Training, the approach yields Swin‑based encoders that outperform or match supervised and self‑supervised baselines on tasks such as monocular depth estimation, 3D object detection, 3D occupancy prediction, and end‑to‑end planning.

By Boshi Li, Xiaohui Wang, Xiaoyang Wu, Zhichao Li, Ya Yang, Naiyan Wang
arXiv Computer Vision
Sep 7

Temporal Residual Neural Radiance Fields for Monocular Video Dynamic Human Body Reconstruction

The paper introduces Temporal Residual Neural Radiance Fields for reconstructing dynamic human bodies from monocular video. It builds a temporal residual field independent of MLPs, reduces trainable parameters, speeds up rendering, and employs a multi‑dimensional loss to improve pixel‑level accuracy. Experiments show higher PSNR and SSIM than recent methods while being roughly 780 times faster than Anim‑NeRF and Neural Body.

By Tianle Du, Jie Wang, Xiaolong Xie, Wei Li, Pengxiang Su, Jie Liu
arXiv Computer Vision
Sep 7

Efficient Multi-Timescale Event Representations for Feed-Forward Object Detection

The paper introduces a confidence‑normalized continuous multi‑timescale representation for event cameras, using logarithmic B‑spline temporal encoding and a geometry‑aware local confidence mechanism. When paired with a fixed feed‑forward EventCenterNet detector, this representation outperforms the compact CSTR representation on the PEDRo and Gen1 datasets. Additionally, a recursive exponential‑polynomial approximation is proposed to allow efficient event‑by‑event updates while maintaining detection performance.

By Fredrik Lundell, Per-Erik Forssen, M{\aa}rten Wadenb\"ack, Astrid Lundmark
arXiv Computer Vision
Sep 7

CLON: Cue-Calibrated Linguistic Object Onboarding for Zero-Shot 6D Pose Front-Ends

CLON (Cue‑Calibrated Linguistic Object Onboarding) is a zero‑shot front‑end for 6D pose estimation that requires no task‑specific training for new objects. It builds a linguistic semantic memory from rendered templates to generate high‑recall proposals and computes fixed cue weights to calibrate proposal scoring. On seven BOP‑Classic‑Core datasets, CLON boosts detection AP by 8.1 pp, segmentation AP by 6.2 pp, and downstream 6D pose AR by up to 4.1 pp compared to existing front‑ends.

By Seojin Ji, Yoojin Kwon, Hyung-Sin Kim
arXiv Computer Vision
Sep 7

Learning Spatial-Spectral Refinement and Calibrating Complementary Observations for Hyperspectral Image Super-Resolution

The paper introduces TSR-ITNR, a two‑stage, self‑supervised framework for hyperspectral image super‑resolution that fuses high‑resolution multispectral and low‑resolution hyperspectral data. Stage 1 refines an implicit Tucker representation using a low‑rank spatial tensor and spectral basis, enhanced by a pretrained denoiser, to capture fine spatial details and spectral correlations. Stage 2 applies parameter‑free calibration to extract complementary corrections from both observations, preserving geometry and ensuring orthogonal complementarity, leading to superior reconstruction quality demonstrated on benchmark datasets and improved downstream segmentation performance.

By Liqian Yang, Xingchi Chen, Xinfeng Gui, Xiangyong Cao, Qianxin Yi
arXiv Computer Vision
Sep 7

Weather-Conditioned Depth Anything

Weather-Conditioned Depth Anything (DA‑W) is a new framework that enhances monocular depth estimation models, like the Depth Anything series, to perform robustly under adverse weather conditions such as fog, rain, snow, and low‑light. It achieves this by disentangling style from content: a Style Filter extracts weather‑specific embeddings from a curated mix of real and synthetic degradation data, which are then injected into the backbone via a lightweight, zero‑initialized adapter. The adapter is trained with pseudo‑label distillation and alignment, enabling a single unified model to adapt to diverse weather scenarios while preserving its generalization on clean data, and it achieves state‑of‑the‑art performance with an average 3.7% improvement in AbsRel on weather benchmarks.

By Zhaoming Xu, Chan-Wei Hu, Kuan-Ru Huang, Zihao Zhu, Renjie Li, Yang Zhou, Zhengzhong Tu
arXiv Computer Vision
Sep 7

Segmentation of the aorta in 4D flow MRI using 4D convolutional kernels and learning from sparse annotations

The study presents a fully automated 4D U‑Net that segments the ascending aorta, arch, and proximal descending aorta in 4D flow MRI using a hybrid 4D convolutional kernel and sparse 4D labels derived from 2D expert contours and centerlines. Trained on 268 scans from eight centers, the model achieved high Dice scores (0.927 internal, 0.911 external) and excellent agreement with expert measurements of peak velocity, net flow, wall shear stress, and diameters (ICC ≥0.954 internal, ≥0.980 external). Compared to frame‑wise 3D networks and semi‑automatic methods, the 4D U‑Net outperformed in diastole and generalised well to independent post‑contrast data.

By Hinrich Rahlfs, Julio Garcia, Chiara Manini, Markus H\"ullebrand, Sebastian Schmitter, Sarah Nordmeyer, Titus K\"uhne, Heiko Stern, Christian Meierhofer, Andreas Harloff, Sebastian Kelle, Alexander Lenz, Peter Bannas, Jeanette Schulz-Menger, Ralf F Trauzeddel, Anja Hennemuth
arXiv Computer Vision
Sep 7

MEOX: Compact Multimodal Mixture-of-Experts for Earth Observation

MEOX is a compact multimodal masked autoencoder designed for Earth Observation that uses a 2.939 million‑parameter encoder and 3.115 million total parameters. It incorporates sensor‑specific adapters, explicit validity signals, and a shared sparse‑expert block to maintain modality‑dependent processing before a learned patch‑wise fusion, followed by fourteen encoder blocks that process a single spatial sequence with four metadata tokens. Pretrained on 1.228 million MMEarth64 samples, MEOX achieves strong performance on GEO‑Bench tasks, surpassing prior CSMoE results, and demonstrates effective sensor‑flexible representation learning with a modest parameter budget.

By Mohanad Albughdadi
arXiv Computer Vision
Sep 7

CrossDepth: Geometry-Constrained Attention for Generalizable Multi-View Surround Depth Estimation

CrossDepth introduces geometry-constrained attention for multi-view surround depth estimation, addressing cross-image inconsistencies caused by varying camera intrinsics and limited receptive fields. The method conditions features on per-pixel camera-aware ray embeddings and extends pixel context via cross-image attention limited to geometrically plausible regions. Trained self-supervised with photometric consistency, it achieves better depth accuracy and consistency on DDAD and nuScenes compared to existing self-supervised approaches.

By Samer Abualhanud, Max Mehltretter
arXiv Computer Vision
Sep 7

LUMIN: Lightweight Universal Manufacturing Inspection Network for Anomaly Detection

The paper introduces LUMIN, a lightweight network for manufacturing anomaly detection, and PSP, a four‑stage adaptive memory bank sampling pipeline that eliminates backbone forward passes. PSP achieves near‑random construction speed, 341× faster than FPS, while parallel similarity computation and stratified pixel sampling cut inference time by 20× with minimal accuracy loss. Experiments on five benchmarks show that LUMIN and PSP reach state‑of‑the‑art sampling accuracy with dramatically reduced latency and memory usage.

By Pengfei Yang
arXiv Computer Vision
Sep 7

From Interpretability Methods to Interpretable Models

The paper argues that explainable AI for computer vision has focused too much on developing interpretability methods rather than assessing how interpretable the models themselves are. It proposes a shift toward model-centric evaluation, using existing tools to compare what different models represent and compute, and emphasizes the need to measure whether humans can truly understand these models. The authors review the current toolbox, survey limited model comparison work, draw parallels to systems neuroscience, and outline a future agenda for model-focused XAI.

By Julien Colin, Nuria Oliver, Thomas Serre
arXiv Computation and Language
Sep 7

KCSAT-ML: Probing Reasoning Models with Nationwide-Cohort Human Difficulty

KCSAT-ML is a new benchmark built from 664 Korean College Scholastic Ability Test mathematics problems, including a 339‑item core set with official per‑item error rates from nationwide cohorts of hundreds of thousands of examinees. The benchmark introduces the Difficulty‑aligned Reasoning Gain (DRG) metric, which evaluates whether a model’s mistakes align with items humans find hard or easy, revealing distinct patterns in how vision‑language and large language models perform across difficulty levels. The dataset and code are publicly available at https://github.com/naver-ai/KCSAT-ML.

By Sanghee Park, Geewook Kim, Kee-Eung Kim
arXiv Machine Learning
Sep 7

LookThere! Sparse Vision by Reinforced Selection

LookThere! Sparse Vision by Reinforced Selection proposes an end‑to‑end reinforcement learning framework that jointly trains a shallow input selector and a deep representation extractor for vision transformers. The selector learns where to focus and the extractor learns what to process, enabling the model to use only a tiny fraction of the input tokens—down to 0.2%—while preserving accuracy. The method outperforms existing selection techniques across diverse tasks and models, including high‑resolution recognition, segmentation, zero‑shot classification, and regression, establishing a new Pareto frontier in performance‑compute trade‑offs.

By Sreehari Rammohan, Yousef Yassin, Anthony Fuller, Junfeng Wen, Carl Vondrick, Evan Shelhamer
arXiv AI
Sep 7

Post Fusion Bird's Eye View Feature Stabilization for Robust Multimodal 3D Detection

The paper introduces Post Fusion Stabilizer (PFS), a lightweight module that refines intermediate bird’s‑eye view (BEV) feature maps in existing camera‑LiDAR fusion detectors. PFS stabilizes feature statistics under domain shift, suppresses regions affected by sensor degradation, and adaptively restores weakened cues via residual correction, acting as a near‑identity transformation. On the nuScenes benchmark, PFS achieves state‑of‑the‑art robustness, notably improving camera dropout robustness by +1.2% and low‑light performance by +4.4% mAP while adding only 3.3 M parameters.

By Trung Tien Dong, Dev Thakkar, Arman Sargolzaei, Xiaomin Lin