The paper introduces a sampling algorithm based on a multivariate Bernoulli distribution to address challenges in multi‑label datasets where labels are non‑exclusive and vary widely in frequency. By estimating distribution parameters from observed label frequencies and computing weights for each label combination, the method produces weighted samples that reflect a target distribution while respecting label dependencies. Applied to Web of Science research articles labeled with 64 biomedical topics, the approach yielded a more balanced sub‑sample, improving representation of minority categories.
arXiv:2606. 05927v1 Announce Type: new Abstract: The complex imbalanced label distribution poses a crucial challenge to multi-label classification, as most classifiers are biased towards the majority class and high-frequent labels.
A key factor in deciding whether to trust an automatic prediction is its confidence score, which should be calibrated to match the actual probability of the prediction being correct. Most confidence c...
arXiv:2609.26468v1 Announce Type: new Abstract: A key factor in deciding whether to trust an automatic prediction is its confidence score, which should be calibrated to match the actual probability o...
arXiv:2607. 22985v1 Announce Type: cross Abstract: Conformalized selection has been widely applied to select high-quality candidates from large datasets with rigorous uncertainty quantification, such as reliable labeling, drug discovery, and the alignment of large language models.
arXiv:2606. 15029v1 Announce Type: new Abstract: LLM judges are used to reduce the need for costly human labor in evaluating open-ended text generation.
The paper examines the common practice of aggregating multiple labels per instance into a single ‘true’ label for supervised learning. By creating a theoretical model, the authors show that using the full, non‑aggregated label information can make it easier to train well‑calibrated models, though the benefits depend on the specific problem. They predict when non‑aggregated labels will improve learning and validate these predictions on real datasets.
arXiv:2606. 07141v1 Announce Type: cross Abstract: Language models trained for clinical disease inference are trained on patient data, which may include sensitive and private information, and data owners may request the removal of their data from a trained model due to privacy or copyright concerns.
arXiv:2601. 15036v4 Announce Type: replace Abstract: Factorizable joint shift (FJS) represents a type of distribution shift (or dataset shift) that comprises both covariate and label shift.
arXiv:2606. 07630v1 Announce Type: cross Abstract: Real-world datasets across image and text domains are often characterized by skewed class distributions and noisy annotations, which jointly degrade model performance, particularly on minority classes.
arXiv:2602. 18518v2 Announce Type: replace Abstract: Content safety teams need metrics that reflect what users actually experience, not only what is reported.
arXiv:2607. 17653v1 Announce Type: cross Abstract: Source-free universal domain adaptation (SF-UniDA) adapts a pre-trained source model to an unlabeled target domain under both covariate and label shifts, without access to source data.
arXiv:2602. 17894v2 Announce Type: replace-cross Abstract: Data collection is a critical component of modern statistical and machine learning pipelines, particularly when data must be gathered from multiple heterogeneous sources to study a target population of interest.