← Back to all news
arXiv Computer Vision By Zibo Zhou, Zongsen Qiu, Rui Chen, Yujie Yao, Yue Zhou, Jianjun Wang

Cross-Architecture Knowledge Distillation from a Vision Foundation Model to a Lightweight Visual State Space Model for Tea Leaf Disease Classification

Read the original on arXiv Computer Vision →

The paper presents a method for cross‑architecture knowledge distillation from a fine‑tuned DINOv2 Vision Transformer teacher to a lightweight bidirectional Visual State Space Model (LVSSM) student for tea leaf disease classification. By addressing training‑stability issues with a progressive convolutional stem and gated selective‑scan block, the 4.45 M‑parameter student achieves a mean test accuracy of 95.41%—a 3.09‑point improvement over the teacher’s 92.32%—while using only one‑fifth of the teacher’s parameters. Ablation studies show that simple logit‑level distillation outperforms intermediate feature alignment, and the gains are specific to students that start below the teacher’s performance.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computer Vision.

Related stories

arXiv AI
Sep 7

Lightweight Vision Transformer Compression for On-Device Plant Disease Detection in Resource-Constrained Agricultural Field Conditions

The paper presents a unified compression framework for Vision Transformers aimed at on‑device plant disease detection in resource‑constrained agricultural settings. It combines Hessian‑Balanced Adaptive Block Pruning, quantization, and attention‑based knowledge distillation, evaluating each component separately before integrating the best performers into a deployment pipeline. On a chilli disease dataset, the compressed models achieve accuracy comparable to the FP32 baseline while reducing model size by 74‑98 %, and the full pipeline attains a 54.5× size reduction to 6.01 MB with 95.13 % accuracy.

By Mahadev Sunil Kumar, Bhavika Gondi, Desaisetty Venkata Satya Sai Swapnith, Gangireddy Rahul Jogi, Sudheesh Manalil, Arnab Raha, Amitava Mukherjee, Parthasarathy Seethapathy, G. Gopakumar
llmscomputer-visionefficiencybenchmarks
More like this →
arXiv Machine Learning
23h ago

Cross-Architecture Foundation-Model Distillation for Edge Flood Segmentation

The paper presents a method for distilling a large 300‑million‑parameter geospatial foundation model (Prithvi‑EO‑2.0) into a compact 0.7‑million‑parameter EfficientViT‑B0 student for flood segmentation. By using the teacher to supervise additional unlabeled Sentinel‑2 imagery, the student’s training set expands without new manual labels, achieving competitive performance on Sen1Floods11 and STURM‑Flood while remaining smaller and faster. After quantization, the student runs as a 1.5‑MB INT8 TensorRT engine on a Jetson Xavier NX, processing 512×512 images in 5.57 ms with ~14 MB of memory.

By Fabian Schmalstieg, Karsten Mueller, Wojciech Samek
computer-visionfine-tuningefficiencybenchmarks
More like this →