The paper introduces idSCD, a white‑box method that identifies training datasets by examining the semantic correlation structure learned by a model. By extracting semantic correlation descriptors (SCDs), the authors can perform dataset‑level membership inference without relying on behavioral or distributional cues. Experiments on natural language inference, emotion classification, and medical text classification show that SCD‑based scores outperform existing black‑box and white‑box baselines, achieving up to a 60% relative gain in ROC‑AUC when datasets exhibit distinct semantic traits.
arXiv:2608. 09209v1 Announce Type: cross Abstract: Neural language models trained on large crowdsourced corpora frequently exploit spurious surface patterns tied to target labels without true linguistic or causal relevance, boosting benchmark performance while failing on adversarial or out-of-distribution inputs.
The paper investigates hallucination detection in black‑box large language models by leveraging two accessible signals: semantic entropy, which captures disagreement among sampled response meanings, and token‑level uncertainty derived from log‑probabilities. It introduces a TopK aggregation technique, a hybrid CoCoA method combining uncertainty with semantic dissimilarity, and two supervised approaches—Gated and Stacked—that integrate token and semantic features. Across seven benchmarks and four language models, the supervised Stacked method performs best in many cases, while TopK and CoCoA remain competitive without labeled data, though all methods require careful threshold calibration.
Neural language models trained on large crowdsourced corpora frequently exploit spurious surface patterns tied to target labels without true linguistic or causal relevance, boosting benchmark performance while failing on adversarial or out-of-distribution inputs. Existing approaches either require manual specification of the feature vocabulary or automate discovery only partially, leaving the gap between dataset-level correlation and model-level exploitation unaddressed.
arXiv:2609.12552v1 Announce Type: new Abstract: Open-vocabulary detection accepts any class list at inference, and promptable segmentation returns regions without class names: the taxonomy has left t...
arXiv:2610.28556v1 Announce Type: cross Abstract: Large language models (LLMs) can leverage semantic feature names alongside summary statistics for tabular feature selection, yet it remains unclear w...
arXiv:2607. 18785v2 Announce Type: replace Abstract: As large language model agents gain access to increasingly large skill libraries, retrieving the right skill becomes critical to reliable capability selection and execution.
arXiv:2607. 18785v1 Announce Type: new Abstract: As large language model agents gain access to increasingly large skill libraries, retrieving the right skill becomes critical to reliable capability selection and execution.
arXiv:2609.22767v1 Announce Type: cross Abstract: The IEEE BigData Cup benchmark combines three prediction problems with different output structures: ordinal suicide-risk classification, multi-label...
The paper introduces Adaptive Dual-Teacher Distillation with Subnetwork Rectification (DDSR), a method for black-box domain adaptation that fuses predictions from a black-box source model and a vision‑language model to generate reliable pseudo‑labels for the target domain. DDSR uses adaptive prediction fusion, subnetwork regularization to prevent overfitting, iterative refinement of pseudo‑labels and prompts, and class‑wise prototypes for self‑training. Experiments on several benchmark datasets show that DDSR outperforms existing state‑of‑the‑art approaches, even those that have access to source data or source model parameters.
arXiv:2610.16152v1 Announce Type: cross Abstract: Although post-training significantly enhances model performance, the use of massive undisclosed data raises concerns about benchmark contamination an...
arXiv:2607. 22766v1 Announce Type: cross Abstract: The alignment of Large Language Models (LLMs) is increasingly bottlenecked by data quality.
The paper proposes a method for discovering interpretable, decision‑relevant features from unstructured data by leveraging large language models (LLMs). It does so by conducting pairwise comparisons of items, asking an LLM to explain the distinguishing features, and then applying a statistical choice model to aggregate these explanations into feature relevance scores. Experiments on a synthetic hiring dataset and the Reddit ChangeMyView debate dataset demonstrate that the approach outperforms direct LLM feature extraction and matches baseline predictive accuracy while providing interpretable insights.