arXiv:2608.22887v1 Announce Type: new Abstract: Large language models (LLMs) are entering decisions in triage and lending, where task-relevant inference must be distinguished from impermissible proxy...
The paper "Medical Causal Hypothesis Verification with Large Language Models" reports a small-scale study evaluating eight LLMs on 17 medical causal hypotheses. The authors introduce an evaluation framework and annotate 1,067 evidence points across six criteria, using nine metrics to assess performance. Results show that while LLMs have strong recall, they frequently fail to provide valid scientific articles, evidence, or reject unsupported hypotheses, revealing a critical limitation for their use in healthcare.
The study evaluates 12 instruction‑tuned open‑weight LLMs on six causal‑graph benchmarks, testing five prompting strategies and four confidence sources. Findings show that LLMs tend to over‑predict edges, misclassify indirect or reversed edges as direct, and exhibit high over‑confidence, while conventional confidence estimates are unreliable and agreement signals offer limited improvement. The results suggest LLMs should be used as externally validated soft causal priors rather than definitive causal‑structure evidence.
The paper investigates whether language models still encode occupational biases even when they appear unbiased in behavioral tests. Using a causal framework, the authors separate bias into internal representations of user competence and observable outputs, deriving steering vectors that show these representations influence model behavior in question‑answering and hiring tasks. Across several open‑weight models, demographic factors such as gender, race, and socioeconomic status affect the models’ internal competence representations, revealing hidden bias that behavioral metrics alone may miss.
The paper examines whether removing declared language fields from de‑identified résumés eliminates demographic leakage in large language models. By keeping language attributes identical and varying only unstructured prose across five ethnocultural groups and three cue‑salience levels, the authors find that non‑language text still allows target‑group recovery (average 0.757, reaching 1.000 under high salience). They also show that evaluation design—such as allowing or forbidding ties—dramatically affects LLM‑as‑a‑judge outcomes, underscoring the importance of evaluation protocol in bias audits.
The paper discusses how large language models (LLMs) can be fine‑tuned with observational data to improve alignment with human preferences and business goals. It highlights that directly using such data can cause models to learn spurious correlations, and introduces DeconfoundLM, a method that removes known confounders from reward signals. Experiments show that DeconfoundLM better recovers causal relationships and outperforms baseline methods by over 16% in objective score when confounding is present.
arXiv:2510.07231v5 Announce Type: replace Abstract: Socio-economic causal effects depend heavily on their institutional and environmental contexts. The same intervention can produce different, even o...
The paper investigates indirect evidence in language models by applying influence functions to identify training sentences that affect predictions on 15 grammatical phenomena from BLiMP. After filtering out direct evidence, the authors select the 60,000 most influential remaining sentences and show that removing them significantly degrades held‑out accuracy, more so than random removal. They further analyze the characteristics of this indirect evidence and demonstrate that clustering phenomena based on shared indirect evidence recovers known linguistic categories.
arXiv:2609.37616v1 Announce Type: new Abstract: Language models tend to agree with whatever a user asserts, and post-training increasingly targets this sycophancy so that models evaluate claims on th...
arXiv:2609.22409v1 Announce Type: new Abstract: Understanding contextual causality is critical for large language models (LLMs), as it enables them to accurately identify causal relations in specific...
arXiv:2606. 29034v1 Announce Type: cross Abstract: Large language models (LLMs) increasingly summarize clinical evidence, where a claim's weight depends on how strongly it is supported.
arXiv:2606. 17165v1 Announce Type: cross Abstract: Organizations and researchers show increasing interest in using large language models (LLMs) in place of human participants in A/B tests, in the hope of experimenting faster and at lower cost.