← Back to all news
arXiv Machine Learning

When Do External Priors Help Low-Resource Structured Prediction? A Ranking-Level Alignment Study

The paper investigates when external priors (teacher models, LLMs, or domain experts) help low‑resource structured prediction, introducing Ranking Prior Alignment (RPA) as a model‑agnostic objective that aligns a model’s ranking distribution to an external prior via a temperature‑scaled KL loss. Experiments on tree‑ensemble and multiple‑instance‑learning tasks show that a prior’s benefit depends on its quality (rank correlation with labels) and the granularity of alignment: high‑quality priors improve performance in cross‑sample tree alignment but can hurt within‑bag MIL alignment, while zero‑information priors act as entropy regularization. The authors propose a two‑factor decision rule and a protocol to separate information transfer from regularization.

Related stories

arXiv Machine Learning
Oct 9

Ranking Prior Alignment for Credit Risk Modeling: When Do External Priors Matter?

The paper introduces Ranking Prior Alignment, a model‑agnostic framework that injects external ranking priors—derived from domain experts, teacher models, or large language models—into credit‑risk scoring models via a temperature‑scaled KL divergence loss. It unifies neural (MIL attention) and tree‑based (XGBoost) architectures under a single loss formulation, requiring no external model at inference and tolerating significant annotation noise. Experiments on a large industrial dataset and public Amex data show consistent AUC gains across multiple model families, with the benefits increasing when data, model capacity, or feature quality are limited.

By Qiye Lu, Jiang Ji, Liang Zhang
llmsbenchmarkssafety
More like this →
arXiv Computer Vision
Aug 24

When does fusing hand-crafted knowledge with learned representations pay? A cost-normalized benchmark of stacking, substitution, and interference

arXiv:2608.21098v1 Announce Type: new Abstract: Fusing prior knowledge with data-driven learning is attractive where data is scarce, yet no controlled account says when it helps, is redundant, or har...

By Ahmad AlMughrabi, Albert Clop, Benjamin Busam, Ricardo Marques, Petia Radeva
computer-visionbenchmarks
More like this →
arXiv AI
Oct 12

First-Order In-Context Shapley: Scalable Data Auditing for LLM Alignment and Evaluation

The paper introduces First-Order In-Context Shapley, an inference‑only method that evaluates data utility for large language model alignment by computing the first‑order term of an in‑context cooperative game over semantic k‑NN neighborhoods in O(k) forward passes. It projects log‑likelihood shifts onto a directed influence graph, yielding three signals—Exemplifying Advantage, Exemplified Advantage, and One‑of‑a‑Kind Score—to identify high‑utility demonstrations and rare prompts. The approach reduces compute by 19× compared to Monte‑Carlo Shapley, filters 1–5% of records, and improves error detection and reward accuracy in DPO training and benchmark auditing. whyItMatters":"By enabling scalable, inference‑only data auditing, the method addresses label noise and safety inversions that hinder post‑training alignment of large language models, leading to measurable improvements in model reliability and reward accuracy."

By Yunting Song, Matthew Watson, Moustafa Farid Alzantot, Peter Grabowski, Jun Qin
llmsragreinforcement-learningfine-tuningbenchmarkssafetydata-engineering
More like this →
arXiv Machine Learning
Aug 28

Keeping the Index Open: The Recommendation-Side Cost of Shared Search and Recommendation

The paper investigates the trade‑off of using a shared search‑and‑recommendation index that scores new items purely from features, thereby keeping the index open to unseen items. Experiments on public logs show that a feature‑based tower can match warm‑item performance (Recall@20 0.9595 vs 0.9510) and a lexical baseline, while a full‑catalog check is inconclusive. The study also quantifies the cost of this openness on recommendation quality across several baselines, revealing that exact full‑softmax training improves recall but is impractical at catalog scale.

By Theodore Rogers, Joe Standerfer, Dmitrii Timoshenko, Haoxue Li, Zuhaib Akhtar, Soyoung Yang
More like this →