arXiv AI By Mustafa Talha \.Ilerisoy, Hung Manh Pham, Mathias Funk, Mykola Pechenizkiy, Aaqib Saeed

Zero-Shot Respiratory Sound Classification through LLM-Augmented Audio-Text Alignment

Read the original on arXiv AI →

The paper introduces a framework that aligns self‑supervised respiratory encoders with medical terminology in a shared latent space, enabling zero‑shot respiratory sound classification. By using a medical LLM to generate structured reports from metadata, the method creates dense semantic anchors for contrastive learning, combining a sigmoid‑based contrastive loss with the encoder’s native SSL objective and similarity‑aware negative sampling. On nine tasks across six datasets, the approach achieves a 61.3% mean zero‑shot AUC, outperforming CLAP and Qwen2‑Audio, and reaches the highest linear probing AUC with only 43% of the data used by full‑scale baselines.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI
Oct 27

Meta-CLIP: Metadata-Conditioned Contrastive Learning for Zero-Shot Disease Inference

Meta-CLIP is a metadata‑conditioned contrastive learning framework that incorporates DICOM acquisition information to reweight cross‑modal negatives during medical vision‑language pretraining. By defining a bounded pairwise margin over numerical and categorical DICOM fields, the method emphasizes distinctions among studies with similar imaging conditions while keeping the kernel fixed during optimization. Evaluations on chest radiography and volumetric CT show that Meta‑CLIP improves zero‑shot classification performance, achieving higher macro‑AUROC scores than baseline CLIP models on multiple benchmarks.

By Haozhe Luo, Nirosh Sivanesan, Shelley Zixin Shu, Francesco Nitti, Mauricio Reyes