CARE introduces a constrained attention refinement framework that uses teacher‑student distillation to improve fine‑grained visual classification while maintaining interpretability. The method keeps the final prediction and explanation within a class‑specific attention student, while a training‑only auxiliary teacher reads selected intermediate DINOv2 layers with learnable queries to fuse multi‑level representations and transfer class‑discriminative knowledge. Additional diversity and sparsity regularizers refine the student’s attention heads, reducing redundancy and encouraging compact trait localization, achieving 78.5% Top‑1 accuracy on CUB and demonstrating faithful explanations via insertion and deletion metrics.
arXiv:2610.32442v1 Announce Type: new Abstract: Fine-grained visual classification (FGVC) requires distinguishing between highly similar categories by focusing on subtle, localized visual differences...
Understanding how supervision signals shape semantic visual representations remains challenging, particularly in fine-grained, low-data settings. Standard evaluations of vision models rely on downstre...
IronViT proposes a new approach to building efficient generalist vision encoders by first consolidating the knowledge of multiple specialist teachers into a softmax attention bridge and then transferring this consolidated representation to a hybrid softmax‑linear attention architecture. This two‑stage distillation process, supported by a curated data pipeline, allows the model to capture semantic, spatial, language‑aligned, and action‑relevant cues while avoiding the high‑resolution cost of traditional softmax attention. Across tasks such as recognition, retrieval, dense prediction, multimodal understanding, and robotic learning, IronViT matches or exceeds the performance of leading specialist and generalist encoders, with the hybrid encoder offering increasing efficiency at higher resolutions.
arXiv:2610.28051v1 Announce Type: cross Abstract: Understanding how supervision signals shape semantic visual representations remains challenging, particularly in fine-grained, low-data settings. Sta...
arXiv:2607.09086v2 Announce Type: replace Abstract: We present Subtoken Vision Transformer (SubViT), a selective image tokenization method for fine-grained visual recognition. Standard Vision Transfo...
arXiv:2605.07191v2 Announce Type: replace-cross Abstract: A recent work shows that Attention Transfer, which transfers only the attention patterns from a pre-trained teacher Vision Transformer (ViT)...
arXiv:2610.09802v1 Announce Type: cross Abstract: Concept-based vision models represent images through an intermediate layer of human-inspectable concepts, so what a model relies on can be traced to...
DeCO: Discriminative Evidence Composition for Fine-Grained Dataset Distillation introduces a method that compresses large training sets into compact synthetic sets while preserving fine-grained visual classification cues. It uses attention rollout from a pretrained TransFG teacher to locate informative patches, applies spatial diversification to avoid redundancy, and organizes these patches into class-wise evidence banks that are packed into grid-composed images. Experiments on CUB-200-2011, FGVC-Aircraft, and Stanford Cars demonstrate that DeCO outperforms existing coreset and dataset-distillation baselines across various images-per-class budgets.
arXiv:2610.02117v1 Announce Type: cross Abstract: On-policy self-distillation has recently emerged as an effective approach for improving language-model reasoning by supervising students with a froze...
The paper introduces CODI, a method that enhances self‑supervised Vision Transformers by mixing a small portion of the teacher’s class‑token distribution into each teacher patch‑token target. This cross distillation approach gives every masked patch an attention‑independent semantic correction without altering the architecture. Experiments demonstrate that CODI improves performance on both dense prediction and image‑level tasks under aligned pretraining settings.
The paper introduces Vision‑RL2, a region‑level reinforcement learning approach that optimizes a lightweight proposal network for fine‑grained multimodal large language model (MLLM) perception. By treating coherent image regions as actions and scoring them with a frozen MLLM reader, the method selectively focuses visual resolution on evidence, reducing token usage while improving accuracy across multiple benchmarks and backbones. The approach eliminates the need for region annotations, response sampling, or reasoning trajectories, and the refined proposals enable sparse encoding that magnifies relevant evidence.
arXiv:2607. 14703v1 Announce Type: cross Abstract: Multiple instance learning (MIL) has become the main paradigm for whole-slide image (WSI) analysis in computational pathology.