arXiv Computation and Language
2d ago

ODC: Orthogonal Drift Correction for Improved Text-to-Image Semantic Alignment at Inference

The paper introduces Orthogonal Drift Correction (ODC), an inference-time guidance method that mitigates semantic drift in text-to-image generation without retraining or extra user input. ODC operates in two stages: first it detects drift by comparing the generated image to the prompt in a shared vision-language embedding space, isolates the orthogonal component of the drift, and converts it back into text; second it refines the text conditioning for a second generation pass using an adaptive rank-reduced concept removal module. Experiments show ODC improves prompt-image alignment, raising CLIP scores by up to 5.4% on COALBench-300 and PartiPrompts, and boosting the 3-in-1 score by 4.5% on T2I-CompBench’s complex subset.

By Kasra Kamali, Basim Azam, Tuan Duong Trinh, Jean Honorio, Naveed Akhtar
arXiv Computer Vision
Oct 28

Training-Free Inference Acceleration for Video Large Language Models: Methods, Trade-offs, and Evaluation

The paper surveys training‑free methods to accelerate inference in Video Large Language Models (VLMs), aiming to reduce computational, latency, and memory costs while retaining task‑relevant spatiotemporal evidence. It categorizes techniques by their intervention points in the inference pipeline—input, visual encoding, projector, prefill, and decoding—and discusses post‑training quantization as a cross‑stage approach. The authors analyze how the position of each intervention influences evidence preservation and computational savings, noting that local efficiency gains may not lead to overall acceleration, and they provide quantitative comparisons from prior work and new experiments to evaluate accuracy‑latency trade‑offs.

By Yuqi Li, Chuanyue Yan, Neil Rodrigues, Xianglong Wang, Xiaoqin Feng, Junming Liu, Xi Xiao, Zhongwei Cheng, Wei Li, Lin Chen, Yingli Tian
arXiv Computer Vision
Sep 4

CoFiE: Coarse-to-Fine Evidence Selection for Efficient Streaming Video Understanding

CoFiE introduces a two‑stage evidence selection framework for streaming video understanding, separating a coarse, query‑agnostic filtering of visually distinctive frames from a fine, query‑specific refinement during LLM prefill. By filtering out redundant frames before expensive vision encoding, CoFiE reduces end‑to‑end latency while maintaining high accuracy. The method achieves state‑of‑the‑art performance on benchmarks such as StreamingBench and OvO‑Bench, improving accuracy by up to 3.15% and inference speed by up to 2.54× compared to prior approaches.

By Jing Jiang, Yiran Ling, Ruonan Li, Dimitrios Stamoulis, Jie Liu