Computer vision

Detection, segmentation, depth and recognition research, plus the vision backbones that keep displacing the last generation.

3,426 stories · RSS feed

arXiv AI
Sep 4

The impact of phase information for few-shot fine-grained image classification

The paper investigates few-shot fine-grained image classification and emphasizes the importance of phase information for capturing structural relationships. It introduces a plug‑and‑play amplitude‑phase integration (API) module that merges local and global frequency amplitude and phase data to create richer feature descriptors. A new network, PSF‑Net, adaptively fuses phase‑based spatial and frequency information and can be integrated into standard episodic training pipelines, achieving superior performance on five public datasets.

By Ruiling Liu, Linyue Zhang, Wenyi Zeng, Jiamiao Lu, Weichuang Zhang, Changming Sun, Zejun Zhang, Xiao Zhao
arXiv Computer Vision
Sep 4

Beyond Small Patches: Black-Box Detection and Purification of Diverse Backdoor Triggers

The paper introduces TRIM, a black‑box defense for backdoor attacks in computer vision models. TRIM identifies and removes malicious trigger regions at inference time using region‑based segmentation, adaptive trigger discovery via inpainting and diffusion, and selective purification, without needing model internals, training data, or clean samples. Experiments on various datasets and trigger types show TRIM reduces attack success rates to as low as 1.16% while maintaining high clean accuracy.

By Ahmed Abdelnaby, Mohamed Elmahallawy
arXiv Computer Vision
Sep 4

Exemplar: Classical Priors Complement Frozen Features for Few-Shot Microscopy Segmentation at Native Resolution

Exemplar is a few‑shot segmentation method that combines a frozen DINOv3 backbone with a fixed bank of classical native‑resolution filter responses in a single lightweight head. Trained only from support masks, it achieves a mean foreground intersection‑over‑union of 0.782 across eleven biomedical imaging datasets, outperforming either component alone and surpassing five other few‑shot methods in 54 of 55 comparisons. With a single annotated mask, Exemplar reaches 0.703, higher than a from‑scratch nnU‑Net trained on the same mask, and while nnU‑Net eventually overtakes it with eight masks, it requires 16–77× longer to fit.

By Michal Pr\r{u}\v{s}ek, Adam Novoz\'amsk\'y, Filip \v{S}roubek
arXiv Computer Vision
Sep 4

ARCOS: Zero-shot Boundary Localization for Corneal Layer Segmentation Across Optical Coherence Tomography Devices

ARCOS is a patch-based zero-shot boundary localization framework designed for corneal layer segmentation in optical coherence tomography (OCT) images. It predicts boundary heatmaps from overlapping native-resolution patches, stitches these predictions across full B-scans, and refines them with a self-conditioned module to produce continuous, anatomically ordered layer segmentations. Evaluated on clinical OCT data from multiple devices, ARCOS achieved 95.1% off‑by‑one boundary accuracy on matched-device tests and 84.3% on unseen devices, with low mean absolute boundary errors and accurate thickness estimates.

By Nuno Vivas Br\'as, Benjamin Memmi, Ma\"elle Bouhassane, Cristina Georgeon, Vincent Borderie, Karsten Plamann, Anatole Chessel
arXiv Computer Vision
Sep 4

ProgResViT: Progressive Resolution and Width for Adaptive Vision Transformers

ProgResViT is an input‑adaptive Vision Transformer that processes images progressively across multiple rounds, starting with a low‑resolution image and a narrow subnetwork and refining the prediction with higher resolution and a wider subnetwork if needed. The method introduces Progress‑Conditioned Soft Gating (PSG) to share a single backbone across rounds while conditioning token fusion and layer outputs on the current round, block, and input resolution. Experiments on DeiT show improved accuracy‑compute trade‑offs compared to adaptive‑width, adaptive‑depth, and dynamic‑token baselines, and the design also benefits self‑supervised DINO representations and downstream semantic segmentation.

By Ali Hojjat, Janek Haberer, Olaf Landsiedel
arXiv Machine Learning
Sep 4

Understanding Autonomous Driving Datasets by Describing Differences between Image Subsets in Natural Language

The paper introduces set difference captioning for autonomous driving datasets, aiming to generate natural‑language descriptions of differences between two image subsets. It adapts a two‑stage approach to focus on object‑centric patches, allowing attribution of differences to specific objects or categories. A new benchmark, AD‑Diff Bench, is presented to evaluate these methods, especially for sparse, real‑world differences, with open‑weight models to ensure reproducibility.

By Julian Truetsch, Felix Hauser, Christoph Stiller, Frank Bieder
arXiv Computer Vision
Sep 4

Improving Clinical Target Volume Segmentation Accuracy using Anatomical Priors and Active Learning for the AGITG TOPGEAR Clinical Trial

The study explores how adding anatomical priors and active learning can improve the accuracy of deep learning models for segmenting the Clinical Target Volume (CTV) in gastric cancer radiotherapy. Using 100 retrospective CT scans, an nnU‑Net model trained on 10 expert‑contoured cases was enhanced with voxel‑wise anatomical prior maps and iterative active learning over four rounds. The combined approach raised the mean Dice Similarity Coefficient from 0.84 to 0.87, demonstrating that both techniques individually and together improve segmentation performance and generalizability.

By Phillip Chlap, Mark Lee, Trevor Leong, Matthew Field, Jason Dowling, Hang Min, Julie Chu, Jennifer Tan, Phillip K. Tran, Tomas Kron, Annette Haworth, Martin A. Ebert, Shalini K. Vinod, Lois Holloway
arXiv Computer Vision
Sep 4

Thermal Imaging for Contactless Cardiorespiratory and Sudomotor Response Monitoring

The paper investigates the use of thermal infrared video to non‑invasively monitor cardiorespiratory and sudomotor activity in industrial human‑machine interfaces. It presents a signal‑processing pipeline that tracks facial regions, aggregates thermal signals, and separates slow sudomotor trends from faster heart‑rate and breathing‑rate components. Experiments on 31 driver‑monitoring sessions show that thermal imaging can estimate heart rate, breathing rate, and electrodermal activity with reasonable accuracy, while highlighting challenges such as ROI selection, polarity changes, latency, and subject variability.

By Constantino \'Alvarez Casado, Mohammad Rahman, Sasan Sharifipour, Nhi Nguyen, Manuel Lage Ca\~nellas, Xiaoting Wu, Miguel Bordallo L\'opez
arXiv Computer Vision
Sep 4

$K$-NeAS: Scalable Multi-Material CT Reconstruction Using Neural SDFs

The paper introduces $K$-NeAS, a scalable neural architecture for multi-material CT reconstruction that replaces separate material networks with a shared latent backbone and a differentiable $K$-material soft selector. It automates attenuation bounds using a Gaussian Mixture Model and adds a scheduled auxiliary floater loss to reduce geometric hallucinations in sparse-view settings. Evaluated on four clinical CBCT datasets, $K$-NeAS achieves higher 3D volumetric fidelity—up to a 1.88 dB PSNR gain over a single-material baseline—and shows improved robustness under extreme sparsity, outperforming baselines by up to 1.17 dB.

By Daksh K. Shah, Emmanouil Nikolakakis, Razvan Marinescu
arXiv AI
Sep 4

ENEAS: Embedding-guided Neural Ensemble for Adaptive Segmentation

ENEAS is a unified, text‑promptable method that simultaneously provides precise instance tracking and high‑quality segmentation, and enables open‑concept discovery of any instance named by a text query. It extends the SeC architecture with a text‑prompting adapter and temporal memory to maintain targets through disappearance and avoid drifting, while a semantic verification layer combines visual embedding matching with conditional VLM refinement to filter ontological errors. Designed for 3D reconstruction, ENEAS delivers robust semantic tracking and segmentation across videos, libraries, and unordered collections, distinguishing true instances from look‑alike doppelgangers.

By Javier del Pino (SperidLabs), Salvador Rodr\'iguez (SperidLabs), Alejandro Garabito (SperidLabs), Javier \'Alvarez (SperidLabs), Chema Garabito (SperidLabs)
arXiv Computer Vision
Sep 4

VeriPhy: Agentic Physical Reasoning for World Model Evaluation and Refinement

VeriPhy is an auditable physical‑verification system that transforms a text prompt into typed physical obligations and a statically validated execution plan before any video frames are generated. During execution, it gates calls to frozen low‑level experts (segmentation, tracking, counting, depth, OCR, audio‑event detection, etc.) and records provenance‑carrying evidence for each action. The system maps these records to a three‑valued state—supported, contradicted, or unknown—providing traceable verdicts that can be used to refine generation models.

By Wenzhuo Xu, Yuchen Zhu, Chongjian Ge, Xuan Shen, Jing Shi, Jason Kuen, Yongxin Chen, Molei Tao, Christopher McComb, Noelia Grande Guti\'errez, Jiuxiang Gu
arXiv Machine Learning
Sep 4

Position: Unlabeled IS NOT Equal to No Human Supervision in Visual Learning

The paper argues that unlabeled data does not mean there is no human supervision in visual learning. It highlights that different data curation and training objectives embed distinct human priors, making the blanket term "unsupervised" misleading. The authors call for explicit disclosure of these priors to improve clarity, comparison, and methodological diversity in the field.

By Dong Lao
arXiv Computation and Language
Sep 4

KhatianDoc: A Human-Verified Benchmark Diagnosing Multimodal LLM Failure on Bengali Legal Land Records

KhatianDoc is a new benchmark that tests multimodal large language models on Bengali legal land records, specifically the handwritten RS Khatians used in Bangladesh. The benchmark comprises four tasks—symbol recognition, base‑16 to decimal conversion, structured field extraction, and legal document question answering—drawn from 107 real records and 1,634 QA pairs. Six multimodal LLMs were evaluated under a zero‑shot protocol, revealing that many models fail to answer a significant portion of questions correctly and perform poorly on arithmetic conversion, highlighting a lack of capability rather than a performance gap.

By Tasmiad Hasan, Arafat Zaman Ratul, Sarker Sadman Saalim, S. M. Shah Nawaz Hossain, Khan Raiyan Ibne Reza, Sumaiya Tabassum Nimi
arXiv Computer Vision
Sep 4

Explainable Convolutional Neural Networks for Retinal Fundus Classification and Cutting-Edge Segmentation Models for Retinal Blood Vessels from Fundus Images

The paper presents a two‑pipeline framework for retinal fundus analysis that combines four‑class disease classification with vessel segmentation. It fine‑tunes eight ImageNet‑pretrained CNNs on the FIVES dataset, applies five gradient‑based explanation methods to assess model interpretability, and benchmarks ten U‑Net variants—including transformer‑based and attention‑enhanced architectures—on the FIVES and DRIVE datasets. The best classification results come from ResNet101 (94.17% accuracy), while the strongest segmentation performance is achieved by Attention U‑Net with a ResNet101V2 backbone, improving DRIVE IoU from 60.80% to 64.83%.

By Fatema Tuj Johora Faria, Mukaffi Bin Moin, Pronay Debnath, Asif Iftekher Fahim, Faisal Muhammad Shah
arXiv Computer Vision
Sep 4

FoRIS: Progressive Foreground Refinement for Training-Free In-Context Segmentation

FoRIS is a training‑free in‑context segmentation framework that refines foreground masks through a coarse‑to‑fine process. It operates in three stages—Foreground Purification, Localization, and Consolidation—to suppress background noise, pinpoint target regions, and reconstruct complete foreground structures. The method achieves state‑of‑the‑art performance, improving mIoU by 4.5 and 4.8 points in 1‑shot and 5‑shot settings respectively.

By Ming Hu, Jianfu Yin, Mingyu Dou, Miaomiao Zhang, Yao Wang, Cong Hu, Bingliang Hu, Quan Wang
arXiv Computer Vision
Sep 4

Efficient Semantic Understanding from Digital Foveation

The paper proposes a lightweight active‑vision pipeline that mimics biological foveation to perform semantic segmentation more efficiently. By selecting salient fixations, observing high‑resolution foveal patches, and using low‑resolution context, the method achieves 95.9% of baseline Top‑1 accuracy with only 4.7% of the computational cost, and recovers 90.6% of baseline object recall using 58.6% of the computation. The results demonstrate that sparse, selectively allocated observations can yield substantial semantic understanding, suggesting active vision as a viable alternative to uniform dense processing.

By Caterina Caccavella, Vittorio Fra, Andreas Ziegler, Giulia D'Angelo, Yulia Sandamirskaya
arXiv Computer Vision
Sep 4

DropClick: Semi-Automated One-Click Segmentation for Agricultural Robotic Data

DropClick is a click‑guided segmentation tool designed to ease the annotation of agricultural vision datasets. It uses single‑click inputs to generate pseudo‑labels, reducing the need for a click on every object and achieving high mIoU scores on SB20 and BUP20 datasets. The method also serves as a pseudo‑labelling approach that can train a Mask2Former model with significantly less user input while maintaining performance.

By Patrick Zimmer, Michael Halstead, Chris McCool
Hugging Face Trending Papers
Sep 3

Efficient Semantic Understanding from Digital Foveation

The paper proposes a lightweight active‑vision pipeline that mimics biological foveation to perform dense semantic segmentation more efficiently. By selecting salient fixations, it uses high‑resolution foveal views, low‑resolution context, and adaptive computation to accumulate semantic information. On ADE20K‑Object, a single foveated observation attains 95.9% of baseline Top‑1 accuracy with only 4.7% of the computational cost, and semantic accumulation recovers 90.6% of baseline recall using 58.6% of the computation.

Hugging Face Trending Papers
Sep 3

Pushing the (Decision) Boundaries: Dynamically Calibrating Differentially Private Noise to Explainability in Federated Learning

The paper introduces XCal-FL, a federated learning algorithm that dynamically calibrates differential privacy noise using three signals—prediction logit variations, counterfactual margins, and saliency concentration—to improve both predictive accuracy and explanation fidelity. Experiments on medical imaging datasets demonstrate that XCal-FL outperforms static-noise and state‑of‑the‑art adaptive DP methods, achieving over 10% better accuracy and up to five‑fold higher explanation fidelity while using privacy budgets more efficiently. The study reveals that explanation fidelity behaves non‑linearly with privacy loss, indicating that explainability is a separate dimension of the privacy trade‑off that cannot be inferred from utility alone.

Hugging Face Trending Papers
Sep 3

Understanding Autonomous Driving Datasets by Describing Differences between Image Subsets in Natural Language

The paper introduces set difference captioning for autonomous driving datasets, aiming to generate natural‑language descriptions of differences between two image subsets. It adapts a two‑stage approach to focus on object‑centric patches, enabling attribution of differences to specific objects or categories. A new benchmark, AD‑Diff Bench, is presented to evaluate this method, especially for sparse, real‑world differences, and the authors provide open‑weight models and code for reproducibility.