arXiv Computer Vision
ReuScope: Auditing Data Reuse and Evaluation Sensitivity in Capsule Vision 2024
Read the original on arXiv Computer Vision →The Flow has not summarised this story yet — read it at arXiv Computer Vision.
Related stories
arXiv Computer Vision
1d ago
Different Models, Shared Failures: Auditing Frozen-Encoder Pools for Medical Segmentation
The paper introduces CoFailMed, an audit framework that examines correlated failures among pretrained encoder pools used for medical image segmentation. By evaluating 11 public encoders across five 2D tasks with frozen features and 1×1 readouts, the study finds that models sharing checkpoints exhibit the highest per‑case Dice correlation, and that distinct checkpoints within the same encoder family are more correlated than cross‑family pairs. The analysis also shows that richer readouts and fine‑tuning can narrow but not eliminate these correlations, and that effective pool size at a 33rd‑percentile Dice threshold varies with encoder family and training recipe, underscoring the need to report absolute dependence and threshold‑specific effective size alongside accuracy.
arXiv AI
Jul 17
Demographically-Conditioned Synthetic Medical Images for Bias Mitigation and Bias Detection in Disease Classifiers
arXiv:2607. 14984v1 Announce Type: new Abstract: Per-subgroup fairness audits of medical image classifiers face a sample-size problem: minority subgroups in held-out test sets have so few samples that the resulting confidence intervals on per-subgroup performance are wider than the bias the audit is meant to detect.
arXiv Computer Vision
6d ago
Readout and Operating-Point Sensitivity in Event-Stream Candidate Selection
The paper evaluates how readout settings and operating points affect event-stream candidate selection in event-triggered recording. Using 520 DailyDVS‑200 clips and 47 participants, the authors compare several encoder configurations—including DINO‑CLS, patch mean, and MobileNetV3‑Small—across different accumulation durations and view counts, reporting metrics such as target capture, false triggers, and partial AUC. The study finds that lightweight baselines and specific operating points significantly influence performance, while no clear advantage is observed for the JEPA‑specific method.
arXiv AI
Jun 10
A Controlled Audit of Pretraining Contamination in Public Medical Vision-Language Benchmarks
arXiv:2606. 10066v1 Announce Type: cross Abstract: Medical vision-language models (VLMs) are evaluated on public benchmarks whose images and question-answer pairs have been freely downloadable for years, yet reported accuracy assumes these examples were absent from pretraining.
arXiv AI
Oct 2
Model validation in machine learning: A scenario-based guide from hold-out splits to nested group cross-validation in biomedical and applied research
arXiv:2610.01284v1 Announce Type: cross Abstract: Model validation estimates the performance of a complete learning procedure on new data. However, an invalid split can produce an optimistic and stab...
arXiv AI
Oct 15
Audit Before You Update: Recording Artifacts and the Limits of Bounded Model Updates in MIMIC-IV ICU Prediction
The paper investigates how to evaluate bounded model updates for ICU prediction models using the MIMIC‑IV database. After auditing the data for recording artifacts and cleaning it, the authors compare a bounded update that retrains only treatment‑context and fusion parameters with full fine‑tuning across two model families and two targets, finding that bounded updates are non‑inferior or superior in most cases but depend on target, model family, and learning‑rate tuning. The study also shows that audits should precede update evaluation and that the update boundary should be chosen per target.