arXiv AI
CreateScore: Domain-Theory-Informed Bayesian Routing for LLM-Based CV Screening
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv Machine Learning
Jul 1
Calibration, Not Compilation: Detecting and Repairing Misspecified Probabilistic Programs Written by Language Models
arXiv:2606. 31630v1 Announce Type: new Abstract: Language models increasingly write probabilistic programs (in NumPyro, Stan, or Pyro), but a program that compiles, runs, and passes every unit test can still be \emph{statistically} wrong -- a Gaussian likelihood for heavy-tailed data, a Poisson for over-dispersed counts, an invalid prior support, or a pathological parameterization.
arXiv Computation and Language
Aug 27
CaSKG: Counterfactual-Causal Skill Graphs for Scalable Agent Skill Retrieval
CaSKG introduces a counterfactual‑causal skill graph framework that calibrates procedural relations before retrieval, building a high‑recall directed candidate graph from semantic, lexical, input/output, and structural evidence and refining it with repair evidence and optional LLM judgment. The framework applies direction‑conditioned textual counterfactual probes—removing, substituting, and reordering skill pairs—to aggregate evidence with Bayesian smoothing, producing a state‑filtered weighted graph for task‑conditioned expansion. Evaluated across six LLM backbones on ALFWorld and ScienceWorld, CaSKG outperforms existing Graph‑of‑Skills methods, improving macro‑average scores and reducing mean environment steps while preserving essential skill dependencies.
arXiv AI
Aug 26
From Causal Plausibility to Causal Reliability: Evaluating LLMs as Calibrated Direct Causal-Edge Classifiers
The study evaluates 12 instruction‑tuned open‑weight LLMs on six causal‑graph benchmarks, testing five prompting strategies and four confidence sources. Findings show that LLMs tend to over‑predict edges, misclassify indirect or reversed edges as direct, and exhibit high over‑confidence, while conventional confidence estimates are unreliable and agreement signals offer limited improvement. The results suggest LLMs should be used as externally validated soft causal priors rather than definitive causal‑structure evidence.
arXiv AI
Aug 5
When Many Answers Are Valid, Voting Fails: Symbolic Verification for Best-of-K Causal Reasoning in LLMs
arXiv:2608. 03506v1 Announce Type: new Abstract: Self-consistency assumes the most frequent answer among sampled reasoning traces is the most reliable, but this can fail in causal reasoning: samples often repeat the same confounding error, and votes fragment across multiple valid answers, letting an invalid answer win despite a valid minority trace.
arXiv AI
Aug 18
Calibrated Trust, Not Sharper Prediction: An Empirical Test of Uncertainty Fusion
arXiv:2608. 14617v1 Announce Type: cross Abstract: A recurring proposal in legal AI is to improve case-outcome prediction by fusing uncertainty tools (evidence graphs with belief propagation, sequential Bayesian odds updating, Dempster-Shafer combination, and conformal prediction) into one pipeline.
arXiv Machine Learning
Sep 25
BLADE: Distilled LLM Regularization for Calibrated Knowledge Graph Completion
BLADE is a variational model for knowledge graph completion that separates latent truth from graph recording and uses distilled offline language‑model judgments as a frozen teacher regularizer. The model provides calibrated probabilities and epistemic uncertainty through posterior samples, while the teacher is only an optional triage factor during inference. Across five benchmarks, BLADE matches ranking performance and significantly reduces expected calibration error, improving ECE, Brier score, and NLL over several baselines, and shows strong performance under controlled missingness and leakage stress tests.