arXiv AI
Proxy reliance in large language model decisions is uncalibrated to predictive evidence
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
Hugging Face Trending Papers
Aug 24
Proxy reliance in large language model decisions is uncalibrated to predictive evidence
Large language models (LLMs) are entering decisions in triage and lending, where task-relevant inference must be distinguished from impermissible proxy use. Current audits ask whether decisions change...
arXiv AI
Sep 2
Medical Causal Hypothesis Verification with Large Language Models
The paper "Medical Causal Hypothesis Verification with Large Language Models" reports a small-scale study evaluating eight LLMs on 17 medical causal hypotheses. The authors introduce an evaluation framework and annotate 1,067 evidence points across six criteria, using nine metrics to assess performance. Results show that while LLMs have strong recall, they frequently fail to provide valid scientific articles, evidence, or reject unsupported hypotheses, revealing a critical limitation for their use in healthcare.
arXiv AI
Aug 26
From Causal Plausibility to Causal Reliability: Evaluating LLMs as Calibrated Direct Causal-Edge Classifiers
The study evaluates 12 instruction‑tuned open‑weight LLMs on six causal‑graph benchmarks, testing five prompting strategies and four confidence sources. Findings show that LLMs tend to over‑predict edges, misclassify indirect or reversed edges as direct, and exhibit high over‑confidence, while conventional confidence estimates are unreliable and agreement signals offer limited improvement. The results suggest LLMs should be used as externally validated soft causal priors rather than definitive causal‑structure evidence.
arXiv Computation and Language
6d ago
EconCausal: A Context-Aware Economic Reasoning Benchmark for Large Language Models
arXiv:2510.07231v5 Announce Type: replace Abstract: Socio-economic causal effects depend heavily on their institutional and environmental contexts. The same intervention can produce different, even o...
arXiv Machine Learning
Sep 14
Aligning Language Models with Observational Data: Opportunities and Risks from a Causal Perspective
The paper discusses how large language models (LLMs) can be fine‑tuned with observational data to improve alignment with human preferences and business goals. It highlights that directly using such data can cause models to learn spurious correlations, and introduces DeconfoundLM, a method that removes known confounders from reward signals. Experiments show that DeconfoundLM better recovers causal relationships and outperforms baseline methods by over 16% in objective score when confounding is present.
arXiv AI
Sep 16
Beyond the Name: Demographic Leakage in De-Identified R\'esum\'es and Evaluation Artifacts in LLM Bias Audits
The paper examines whether removing declared language fields from de‑identified résumés eliminates demographic leakage in large language models. By keeping language attributes identical and varying only unstructured prose across five ethnocultural groups and three cue‑salience levels, the authors find that non‑language text still allows target‑group recovery (average 0.757, reaching 1.000 under high salience). They also show that evaluation design—such as allowing or forbidding ties—dramatically affects LLM‑as‑a‑judge outcomes, underscoring the importance of evaluation protocol in bias audits.