Hugging Face Trending Papers

HiVE-VQA: Hierarchical Interaction via Volumetric Embedding for Video Quality Assessment

HiVE-VQA introduces a new approach to no-reference video quality assessment by leveraging a single frozen JEPA-based video foundation model. The method extracts token grids from multiple depths of a pre‑trained V-JEPA 2.1 encoder, stacks them into a hierarchical spatio‑temporal volume, and fuses them with a lightweight 3D convolutional head. With only 5.2 M trainable parameters, HiVE-VQA outperforms 17 state‑of‑the‑art methods on both intra‑dataset and cross‑dataset benchmarks, demonstrating that depth‑wise hierarchical fusion from a frozen backbone is both effective and efficient.

arXiv Computer Vision
Sep 16

High-Fidelity Video Quality Assessment with VQA-Specific Saliency

High-Fidelity Video Quality Assessment (HFVQA) is a new framework that uses fixed-size spatio‑temporal patches across multiple scales, including the original resolution, to preserve low‑level quality cues and semantic context. It incorporates a lightweight auxiliary network that learns VQA‑specific saliency directly from quality supervision, enabling the model to focus on the most important spatio‑temporal regions. By combining high‑fidelity cues with task‑specific saliency, HFVQA achieves state‑of‑the‑art performance on standard no‑reference VQA benchmarks while processing only about 12% of the candidate patches, making it computationally efficient.

By Hakan Emre Gedik, Shashank Gupta, Alan Bovik
arXiv AI
Sep 4

LRConv-NeRV: Low Rank Convolution for Efficient Neural Video Compression

LRConv-NeRV introduces low‑rank separable convolutions into the NeRV neural video decoder, replacing selected dense 3x3 layers to reduce computational load and memory usage. By applying low‑rank factorization progressively from the largest to earlier decoder stages, the method offers controllable trade‑offs between reconstruction quality and efficiency. Experiments show that applying LRConv only to the final decoder stage cuts decoder complexity by 68% and model size by 9.3% with negligible quality loss, while INT8 quantization preserves performance close to the dense baseline.

By Tamer Shanableh
arXiv Computer Vision
6d ago

Rethinking Evaluation for Video Capsule Endoscopy: Validation Protocol, Source-Video Diversity, and Cross-Dataset Shift

The paper introduces DMSNet, a deformable adaptive multi‑scale edge‑guided attention network for video capsule endoscopy (VCE) classification. It evaluates DMSNet and 18 other architectures on the Kvasir‑Capsule dataset using both random frame‑level and source‑video‑disjoint validation protocols, reporting MCC scores of 0.442 and 0.456 respectively. The study also examines cross‑dataset performance on CV2024, the impact of source‑video diversity, and shows that model rankings and accuracy vary significantly with validation protocol, training source diversity, and external dataset shift.

By Smriti Regmi, Debesh Jha
arXiv Computer Vision
Aug 25

VideoChat3: Fully Open Video MLLM for Efficient and Generalist Video Understanding

arXiv:2607.14935v2 Announce Type: replace Abstract: Recent advances in video understanding have spanned motion, long video, and streaming interaction, driving this field toward real-world application...

By Xinhao Li, Yuhan Zhu, Xiangyu Zeng, Yuhao Dong, Haoning Wu, Zhiqiu Zhang, Yuandong Yang, Changlian Ma, Qingyu Zhang, Yansong Shi, Xinyu Chen, Haoran Chen, Zizheng Huang, Jun Zhang, Kun Ouyang, Lin Sui, Ziang Yan, Yicheng Xu, Chenting Wang, Yinan He, Hongjie Zhang, Yi Wang, Yu Qiao, Yali Wang, Ziwei Liu, Kai Chen, Limin Wang
arXiv AI
Aug 28

LeVJEPA: Efficient & Scalable Video Pretraining without the Heuristics

LeVJEPA is a video encoder that eliminates the need for architectural asymmetries, exponential-moving-average target encoders, stop-gradients, and capacity-limited predictors used in prior self‑supervised methods. It trains a single encoder with an invariance loss over global and local views, regularized by SIGReg to prevent collapse, and achieves strong performance with far less pretraining compute. The approach also allows block‑causal attention, making temporal ordering a property of the encoder itself, and matches or surpasses state‑of‑the‑art baselines on both appearance‑centric and motion‑centric benchmarks.

By Lukas Kuhn, Lucas Maes, Giuseppe Serra, Quentin Le Lidec, Yann LeCun, Randall Balestriero, Florian Buettner
arXiv Machine Learning
Sep 24

ScoutNeRV: Rapid Encoding of Grid-Based Video INRs via ScoutNet

ScoutNeRV introduces a content‑adaptive initialization framework that speeds up the training of hierarchical grid‑based video implicit neural representations (INRs). By using a lightweight scout network to select a pre‑trained expert from a memory bank, the method transfers the expert’s grid and decoder parameters to a target HiNeRV model, achieving a 21.25 dB PSNR boost before fine‑tuning and reaching near‑baseline performance after only 37 epochs. The approach delivers a 9.25× wall‑clock speedup while maintaining competitive rate–distortion performance.

By Naser Alizada, Farhang Baghban, Hashem Pishkar, Ali Mousavi
arXiv Computer Vision
Sep 30

TT-VidT: Decoupling the Temporal Axis for Efficient Motion-Centric Video Pretraining

TT-VidT is a video pretraining method that decouples the temporal axis by combining a per‑frame ViT-B/16 spatial encoder with a compact Temporal Transfer Layer trained via Diff Compression. The authors conduct a systematic 24‑configuration study to isolate architecture, objective, and decoder effects, showing that the full TT-VidT design yields the strongest motion‑sensitive representations. In downstream fine‑tuning, TT‑VidT outperforms state‑of‑the‑art baselines on Jester, Something‑Something V2, ARID, and Diving48 while using significantly fewer encoder FLOPs.

By Shih-Ying Yeh, Daniel Z. Kaplan, Xuehai Wang, Fu-En Yang, Min-Hung Chen, Shang-Hong Lai
arXiv Computer Vision
Sep 7

Scalable Neural Video Representation Compression

Scalable Neural Video Representation Compression (S-NVRC) introduces a scalable implicit neural representation (INR) video codec that supports fine-grained bitrate and decoding‑complexity scalability from a single embedded bitstream. It uses a coarse‑to‑fine prefix for feature grids and a nested prefix for network layers, enabling a wide range of operating points while maintaining a single encoding. On the UVG dataset, S‑NVRC outperforms SHM 12.4 and multi‑layer VTM‑20.0 by 43.7 % and 5.6 % in BD‑rate, respectively, and offers flexible complexity scalability.

By Tianhao Peng, Ho Man Kwan, Fan Zhang, Shan Liu, David Bull
arXiv Computer Vision
Sep 23

DIFTA-3D: Depth-Consistent Instance-Level Feature Transfer and Adaptation of DINOv3 for 3D Detection

The paper introduces DIFTA-3D, a method that replaces the task‑specific visual branch in IIFNet3D with a frozen DINOv3 foundation model for RGB‑D 3D instance detection. It employs a depth‑consistent feature pipeline that projects points into calibrated RGB‑D frames, filters features with a metric depth‑residual check, caches accepted DINOv3 features, and aggregates them within proposal‑aligned RoI grids. Extensive experiments on ScanNetV2 show that the DINOv3 control achieves mAP scores of 76.15/60.93 at IoU thresholds 0.25/0.50, while the Conservative VAID recipe improves these to 76.59/62.16, indicating a modest gain from the proposed transfer recipe.

By Linman Wang, ZiFei Zhang, Chunran Zheng, Xiwang Dong, Jiarong Lin