arXiv:2607. 18465v1 Announce Type: new Abstract: Crowdsourced labeling provides valuable labeled data for domains across natural language processing, computer vision, and video.
Crowdsourced labeling provides valuable labeled data for domains across natural language processing, computer vision, and video. Label aggregation aims to infer latent true labels from noisy and biased annotations, with the key lying in annotator reliability estimation.
The paper introduces VISTA, a baseline for dense multi‑label classroom coding that leverages the COPUS protocol as a video benchmark. VISTA applies MiniCPM‑V‑4.5 over sliding windows, refines predictions with an MLP head, and aggregates results onto a 2‑minute COPUS grid, achieving 80.1% macro accuracy on held‑out chemistry lectures. The authors also identify systematic failure modes and provide benchmark tooling and code on GitHub.
The paper introduces Language-driven Dense Semantic Adaptor (LDSA) for multi-label image classification with incomplete annotations. LDSA leverages multimodal pretrained CLIP models to extract prior-adaptive relationships, employing a densely contrastive adaptor for visual contrastive constraints and a language-driven interactive decoder with class-specific prompt tuning. Experiments show LDSA achieves state‑of‑the‑art performance on public benchmarks and reveals implicit semantic relationships through its learning scheme.
The paper demonstrates that object detection benchmarks suffer from incomplete annotations, with re-annotation of COCO, Pascal VOC, Cityscapes, and KITTI revealing up to a 60% increase in detected objects, especially small, occluded, or densely packed instances. The authors propose a scalable annotation pipeline that uses multiple annotators per object to capture uncertainty and improve recall, and they introduce two new large-scale benchmarks: an uncertainty-aware detection benchmark and a label error detection benchmark based on real errors. Their findings show that benchmark performance is highly sensitive to annotation quality, yet model rankings remain largely unchanged, highlighting the need for uncertainty-aware evaluation to better reflect real-world ambiguity.
arXiv:2604. 17289v2 Announce Type: replace Abstract: Supervised fine-tuning of large language models relies on human-annotated data, yet annotation pipelines routinely involve multiple crowdworkers of heterogeneous expertise.
arXiv:2606. 08718v1 Announce Type: cross Abstract: While Deep Active Learning (DAL) effectively reduces human annotation costs, its efficacy is constrained by human annotation errors.
arXiv:2605. 13801v2 Announce Type: replace-cross Abstract: As generative AI models such as large language models (LLMs) become more pervasive, ensuring the safety, robustness, and overall trustworthiness of these systems is paramount.
arXiv:2607. 16239v1 Announce Type: new Abstract: AI judges offer a scalable, low-cost alternative to human evaluation, but their outputs can be biased relative to human preferences and highly item-dependent, varying across judges, tasks, and domains.
arXiv:2607. 05393v1 Announce Type: cross Abstract: Time-domain surveys generate many transient candidates, making Real-Bogus classification a critical step in automated discovery pipelines.
The paper introduces FailSAE, a method that uses Sparse Autoencoders to predict failures in vision‑language models (VLMs) such as CLIP. By treating failure prediction as a classification over sparse SAE latent activations and employing a three‑stage training pipeline, the approach yields higher prediction accuracy than existing confidence‑score or auxiliary‑classifier baselines. Analysis shows that the SAE captures class‑specific concepts and reveals a shift toward ambiguous or style‑related concepts during failures, offering insights for runtime failure recovery.
arXiv:2607. 24622v1 Announce Type: cross Abstract: We study imbalanced crowdsourcing with a focus on class-dependent annotator accuracy, a setting that, to the best of our knowledge, remains relatively underexplored despite its importance in real-world inspection systems where the labels of greatest operational importance are also the rarest ones.
ReCalMatch introduces a reliability‑calibrated semantic framework for semi‑supervised fine‑grained visual recognition, addressing the problem of overconfident pseudo‑label errors that arise when visually similar categories produce incorrect high‑confidence predictions. The method constructs class‑conditioned semantic prototypes from class names and domain‑specific aspects, and computes a visual‑semantic agreement score to calibrate pseudo‑label reliability alongside prediction confidence and entropy. Experiments on datasets such as CUB‑200‑2011, Stanford Dogs, NABirds, and iNaturalist18 demonstrate that ReCalMatch consistently improves strong SSL baselines, especially in low‑label regimes where pseudo‑label noise is most severe.