The paper presents a method for cross‑architecture knowledge distillation from a fine‑tuned DINOv2 Vision Transformer teacher to a lightweight bidirectional Visual State Space Model (LVSSM) student for tea leaf disease classification. By addressing training‑stability issues with a progressive convolutional stem and gated selective‑scan block, the 4.45 M‑parameter student achieves a mean test accuracy of 95.41%—a 3.09‑point improvement over the teacher’s 92.32%—while using only one‑fifth of the teacher’s parameters. Ablation studies show that simple logit‑level distillation outperforms intermediate feature alignment, and the gains are specific to students that start below the teacher’s performance.
arXiv:2609.07915v1 Announce Type: cross Abstract: Large vision models provide useful representations for remote-sensing segmentation but are often too expensive for deployment at the satellite or fie...
Dyna‑DINO introduces a curriculum for Vision Transformer (ViT) knowledge distillation that uses the teacher’s intermediate feature maps as progressively harder targets, enabling a student to build foundational representations before tackling higher‑level abstractions. The approach accelerates convergence and improves performance across multiple tasks: on ImageNet‑100 the distilled ViT‑S reaches 90.1% accuracy (+12.24% over baseline), while on ImageNet‑1K it yields +3.9% and +6.09% gains on Oxford and Paris retrieval, +1.93% on semantic segmentation, and notable classification improvements. Additionally, the curriculum reduces training FLOPs by 25.1% and training time by 21% on ImageNet‑100 through early‑stopping of teacher inference.
arXiv:2610.32512v1 Announce Type: new Abstract: Distilled text-to-image models generate images quickly, but classifiers trained on those images can perform substantially worse than classifiers traine...
A100 is a hybrid CNN‑Transformer vision foundation model trained on 3 million unlabeled agricultural images using self‑supervised techniques (DINO, iBOT, KoLeo). It achieves superior crop classification (95.37 % with a frozen backbone) and phenotyping accuracy, and after fine‑tuning reaches 99.90 % on 32‑class crop identification and 99.02 % on 44‑class disease detection. With only 1 % of labeled data, A100 attains 98.01 % plant classification accuracy, outperforming existing baselines and other agricultural foundation models by significant margins.
TinyCNN is a lightweight convolutional neural network with only 193,190 trainable parameters designed for on‑device plant disease classification. It achieves 98.88% test accuracy on the 38‑class PlantVillage benchmark, outperforming larger models while consuming far less energy, memory, and cost. The study also explores knowledge distillation to further compress the model and evaluates cross‑dataset robustness, finding a significant performance drop when moving from PlantVillage to PlantDoc due to background‑driven shortcut learning.
arXiv:2606. 14684v1 Announce Type: cross Abstract: Real-time fire classification systems require models that are simultaneously accurate, computationally efficient, and deployable on resource-constrained hardware.
arXiv:2412. 01282v2 Announce Type: replace-cross Abstract: Vision-Language Models (VLMs) bring powerful understanding and reasoning capabilities to multimodal tasks.
The paper presents a unified compression framework for Vision Transformers aimed at on‑device plant disease detection in resource‑constrained agricultural settings. It combines Hessian‑Balanced Adaptive Block Pruning, quantization, and attention‑based knowledge distillation, evaluating each component separately before integrating the best performers into a deployment pipeline. On a chilli disease dataset, the compressed models achieve accuracy comparable to the FP32 baseline while reducing model size by 74‑98 %, and the full pipeline attains a 54.5× size reduction to 6.01 MB with 95.13 % accuracy.
The paper presents a method for distilling a large 300‑million‑parameter geospatial foundation model (Prithvi‑EO‑2.0) into a compact 0.7‑million‑parameter EfficientViT‑B0 student for flood segmentation. By using the teacher to supervise additional unlabeled Sentinel‑2 imagery, the student’s training set expands without new manual labels, achieving competitive performance on Sen1Floods11 and STURM‑Flood while remaining smaller and faster. After quantization, the student runs as a 1.5‑MB INT8 TensorRT engine on a Jetson Xavier NX, processing 512×512 images in 5.57 ms with ~14 MB of memory.
arXiv:2607. 14703v1 Announce Type: cross Abstract: Multiple instance learning (MIL) has become the main paradigm for whole-slide image (WSI) analysis in computational pathology.
arXiv:2608. 15972v1 Announce Type: cross Abstract: Synchronized camera and wireless measurements observe the same scene through different physical channels.
The paper introduces Cross-Scale Channel-wise Knowledge Distillation (CSCWD), a training-time framework that transfers high‑resolution spatial representations from a YOLO11m‑P2 teacher to a lightweight YOLO11n student without changing the student’s inference architecture. CSCWD aligns teacher P2 features with student P3 while also applying same‑scale distillation at deeper pyramid levels, yielding a 2.92‑point mAP@0.5 improvement over the baseline and a 2.09‑point gain over same‑scale distillation alone. In zero‑shot tests on DUT‑Anti‑UAV and on a Raspberry Pi 5, the 2.58‑million‑parameter student reaches 50.32% mAP@0.5 at 82.32 ms latency (12.15 fps) with negligible runtime or memory increase.