← Back to all news
arXiv AI By Shuting Xie, Nathaniel Lesperance, Graham W. Taylor

Hierarchy-Aware Supervised Uncertainty Estimation for Black-box LLM Taxonomic Reasoning

Read the original on arXiv AI →

The paper introduces a method for estimating uncertainty in hierarchical taxonomic reasoning produced by black‑box large language models (LLMs). By extracting proxy features with an open‑source tool and training lightweight supervised estimators that incorporate hierarchy‑aware supervision, the authors predict rank‑wise correctness. Across three LLMs, these estimators outperform token‑likelihood baselines, raising micro AUROC from 0.57 to 0.75–0.80, with a rank‑specific multi‑head design delivering the best results.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv Machine Learning
Sep 24

Active Learning for Biodiversity Monitoring: From Label Efficiency to Reliable Ecological Inference

The paper discusses how active learning (AL) can alleviate the expert annotation bottleneck in biodiversity monitoring by selecting the most informative samples under a fixed budget. It highlights that while AL reduces labeling effort, its non-random sample selection complicates model validation, calibration, and ecological inference, issues often overlooked in current studies. The authors review existing AL research across acoustic and image data, identify gaps such as limited species coverage and lack of real-world deployments, and propose a tutorial framework and roadmap for developing AL methods that support efficient training, reliable validation, and trustworthy ecological conclusions.

By Ben McEwen, Shiqi Zhang, Dan Stowell
multimodalbenchmarks
More like this →
arXiv Machine Learning
1d ago

Uncertainty-Aware Learning from Multi-Expert Interval Targets

The paper introduces a method for learning from multiple experts who provide interval labels, addressing both within‑label imprecision and between‑expert variation. It harmonizes diverse label vocabularies into a shared probabilistic space, retains individual intervals using a mixture of Beta distributions, and decomposes predictive uncertainty into components that are matched to their corresponding sources of label uncertainty. On sea‑ice concentration data, the approach achieves a 31% reduction in mean absolute error compared to hard‑label baselines and outperforms several aggregation and interval‑regression methods.

By Samira Alkaee Taleghan, Younghyun Koo, Andrew P. Barrett, Farnoush Banaei-Kashani
More like this →
arXiv Machine Learning
Sep 23

Targeted Review for AI-Assisted Biodiversity Surveys: Active Continuous-Score Occupancy Modeling

The paper introduces ACORN, a method that blends machine‑learning predictions with occupancy models to guide ecologists in selecting which samples to review. By strategically choosing the most informative labels, ACORN achieves ecological conclusions nearly identical to fully human‑labeled data while dramatically reducing the number of expert reviews needed. The approach is evaluated on camera‑trap and bioacoustic datasets, demonstrating its effectiveness across real‑world biodiversity surveys.

By Timm Haucke, Lauren Harrell, Justin Kay, Mary Clapp, Sara Beery
More like this →