arXiv AI
Aug 11
Epistemic Transfer in AI-Assisted Verification: A Framework and Evaluation Protocol
arXiv:2608. 08882v1 Announce Type: cross Abstract: AI tools that help people judge online claims are usually evaluated while the tool is present.
Epistemic Transfer in AI-Assisted Verification: A Framework and Evaluation Protocol
AI tools that help people judge online claims are usually evaluated while the tool is present. This paper asks a different question: after using such a tool, what can the user still do on their own?
Related stories
arXiv Machine Learning
Jul 1
The Calibration Turn in AI-Assisted Research: A Conceptual and Methodological Framework for Evidence-Licensed Claims
arXiv:2606. 31273v1 Announce Type: new Abstract: AI-assisted research has entered a stage in which the central question is not only whether systems can generate hypotheses, run experiments, or produce manuscripts, but whether their scientific claims are calibrated to the evidence that supports them.
arXiv Machine Learning
Jul 30
When benchmark inferences do not compose: Projectibility in AI evaluation
arXiv:2607. 26159v1 Announce Type: cross Abstract: An AI benchmark result rarely reaches a consequential claim in one step.
arXiv AI
Sep 15
Beyond AI Literacy: A Structured Review and Exploratory Meta-Analysis of Measures for Competent Generative-AI Use
arXiv:2609.15624v1 Announce Type: cross Abstract: Researchers assessing competent generative-AI use at work must choose among self-reports, objective tests, and measures of oversight and reliance. We...
arXiv AI
Sep 21
EnterpriseVal: Quantifying the Efficacy, Reliability and Value of Generative AI in the Enterprise
arXiv:2609.21841v1 Announce Type: new Abstract: Frontier language models now produce professional deliverables that expert graders judge to match human work on a substantial share of economically val...
arXiv AI
Jun 10
Does Capability Transfer to Subjective Behavior -- and Would Our Instruments Tell Us? A Self-Evolving, Trust-by-Construction Evaluation Paradigm
arXiv:2605. 27914v2 Announce Type: replace-cross Abstract: Benchmarking is mature where answers are verifiable -- math, code, reasoning -- but the fastest-growing uses of LLMs are subjective and human-facing: companionship, emotional support, counseling.
arXiv AI
2d ago
Cybernetic and Epistemic: A Missing Vocabulary for Trustworthy Agentic Delegation
The paper argues that as AI systems increasingly generate code, the bottleneck has shifted to supervising these systems, revealing a vocabulary gap between cybernetic coordination (actions aligning with the world) and epistemic coordination (understanding that can be verified). It critiques current oversight that merely approves outputs, proposing instead that every consequential choice by an agent must include a retrievable condition explaining why it was made, enabling third‑party verification. The authors illustrate this with three delegation episodes, introduce a two‑part reconstruction test, and propose the ORRCF convention to embed such conditions in all recorded decisions.
arXiv AI
Sep 11
Scores Alone Do Not Prove Discovery: The Discovery Certification Protocol for Auditing AI Research Agents
The paper introduces the Discovery Certification Protocol (DCP), a framework that transforms claims from AI research agents into executable tests for recovery and feedback. DCP includes multiple gates that validate improvements, provide controlled information, and measure the impact of truthful feedback, while its core requires strict controls and finite‑sample bounds. Controlled audits in SQLite optimization and virtual catalyst control demonstrated zero recoveries across 96 episodes, with rigorous verification by a deterministic, LLM‑free verifier.
arXiv AI
Aug 11
From Trajectories to Evidence: Auditable Experimental Records for Industrial Research Agents
arXiv:2608. 05235v1 Announce Type: cross Abstract: Research agents increasingly conduct multi-round machine-learning experiments in industrial recommendation settings and retain the resulting trajectories to guide later decisions.
arXiv AI
Aug 17
CLAIR-Fin: An Adversarial Multi-Agent Framework for Claim-Level Verification and Adaptive Debate in Cross-Modal Financial QA
arXiv:2608. 13706v1 Announce Type: cross Abstract: Existing defenses against hallucination in retrieval-augmented and multi-agent pipelines remain partial: evidence is trusted despite modality disagreement, debate verifies an aggregate report rather than individual claims, and such verification occurs only after drafting, leaving inter-agent errors undetected until the final text.
arXiv AI
Aug 20
Competence, Not Accuracy: A Diagnostic for Reference-Free Judge Gates in Skill Optimization
The paper introduces a diagnostic for reference‑free judge gates in text‑space skill optimization. It formalizes a judge as a latent solver, deriving a closed‑form bound on discriminability (ROC‑AUC) in terms of judge competence and answer‑space size, and shows that discriminability is confounded by item difficulty unless a within‑question estimator is used. A non‑intervening probe demonstrates that discriminability is at chance near the competence floor, rises above it, and that the diagnostic can predict gating errors in closed‑loop experiments.
arXiv AI
Jul 24
pAI-Econ-claude: A Gated Human-in-the-Loop Multi-Agent Architecture for AI-Assisted Economic Theory Development
arXiv:2607. 21268v1 Announce Type: cross Abstract: In many social-science research tasks, such as economics, LLM-based agents must produce outputs for which no cheap, task-complete, machine-readable correctness signal exists.