arXiv Machine Learning

When Does a Classifier Help an LLM? Classifier-Guided Prompting and Hybrid Classifier-LLM Models for Credit-Default Prediction

arXiv Computation and Language
Aug 28

The Score Granularity Gap in Black-Box LLM Classification: A Comparative Study of Confidence Constructions

The paper investigates the "score granularity gap" in black-box large language model (LLM) classifiers, asking how finely a confidence score can be thresholded for deployment. By comparing seven confidence construction methods across 25 model-dataset pairs, the authors find that single-shot verbalized confidence, when properly converted to a probability, ranks well but offers only a few distinct threshold values, limiting operational flexibility. The study also shows that multi-query aggregation can improve weak models but may harm strong ones, and provides concrete guidance for deployment trade-offs.

By Ao Sun, Tian Sun, Jiaxing Geng
arXiv Machine Learning
Jul 16

Foundation Models for Credit Risk Prediction: A Game Changer?

arXiv:2605. 18147v2 Announce Type: replace Abstract: Predictive models play a pivotal role in credit risk management, guiding critical decisions through accurate estimation of default probabilities and losses.

By Bart Baesens, Andreas Goethals, Stefan Lessmann, Simon De Vos, Cristi\'an Bravo, David Martens, Victor Medina-Olivares, Christophe Mues, Maria Oskarsd\'ottir, Seppe vanden Broucke, Tony Van Gestel, Tim Verdonck, Wouter Verbeke
arXiv AI
Aug 28

Explainable Artificial Intelligence for Customer Churn Prediction in Telecommunications: A Framework for CRM Integration

The paper presents a framework for integrating explainable AI into customer churn prediction for telecommunications. It benchmarks four classifiers—Logistic Regression, Random Forest, XGBoost, and LightGBM—on the IBM Telco Customer Churn dataset, finding comparable performance with Logistic Regression achieving the highest AUC-ROC and LightGBM the highest accuracy. Explanations are provided via SHAP and LIME at both global and instance levels, and a four‑layer CRM integration architecture is proposed to translate risk scores and attribution vectors into actionable retention strategies, projecting a 3.3–5.3 percentage point reduction in churn and $199K–$319K savings per campaign cycle.

By Sandeep Gaddamwar
arXiv Computation and Language
Sep 11

Rethinking Verbalized Confidence for LLM-as-a-Judge: A Compatibility Shift on Post-2025 Proprietary Models

The paper argues that verbalized confidence—once viewed as overconfident and coarse—has become the preferred soft‑scoring method for LLM‑as‑a‑Judge on top‑tier proprietary models released after 2025. Experiments on SummEval, AggreFact, and HelpSteer2 across up to 18 LLMs show that log‑probabilities are no longer the best signal, and that adding an overconfidence advisory and self‑debate further improves calibration and robustness. The authors note that these enhancements incur little accuracy loss on post‑2025 models but do affect pre‑2025 ones, highlighting a compatibility shift in how confidence should be measured.

By Yu-Chung Hsiao
arXiv Machine Learning
Oct 9

Ranking Prior Alignment for Credit Risk Modeling: When Do External Priors Matter?

The paper introduces Ranking Prior Alignment, a model‑agnostic framework that injects external ranking priors—derived from domain experts, teacher models, or large language models—into credit‑risk scoring models via a temperature‑scaled KL divergence loss. It unifies neural (MIL attention) and tree‑based (XGBoost) architectures under a single loss formulation, requiring no external model at inference and tolerating significant annotation noise. Experiments on a large industrial dataset and public Amex data show consistent AUC gains across multiple model families, with the benefits increasing when data, model capacity, or feature quality are limited.

By Qiye Lu, Jiang Ji, Liang Zhang