arXiv Machine Learning
Jul 22
Data Reliability Scoring
arXiv:2510. 17085v2 Announce Type: replace Abstract: How can we assess the reliability of a dataset without access to ground truth?
Position: Every Ground Truth is a Human Construction, not an Objective Truth
arXiv:2607. 09668v1 Announce Type: new Abstract: Ground truth datasets play a fundamental role as reference values in the training and evaluation of machine learning models.
Related stories
arXiv AI
Jul 23
Information Discernment in Large Language Models
arXiv:2607. 19355v1 Announce Type: new Abstract: LLMs are increasingly used with external knowledge sources like the internet.
arXiv Machine Learning
Jun 10
Can we trust our models? Epistemic calibration in second-order classification
arXiv:2606. 10777v1 Announce Type: new Abstract: Uncertainty estimation is critical for deploying machine learning models in high-stakes settings.
arXiv AI
Jul 13
Ceci n'est pas une pipe: AI systems as semantic abstractions
arXiv:2607. 09489v1 Announce Type: new Abstract: An AI system's output is not the fact or world state it appears to describe, but rather an engineered representation.
arXiv AI
Jul 10
The Contribution of XAI for the Safe Development and Certification of AI: An Expert-Based Analysis
arXiv:2408. 02379v2 Announce Type: replace-cross Abstract: Developing and certifying safe - or so-called trustworthy - AI has become an increasingly salient issue, especially in light of upcoming regulation such as the EU AI Act.
arXiv AI
Jun 6
A Framework for Measuring Appropriate Reliance on Set-Valued AI Advice
arXiv:2606. 06081v1 Announce Type: new Abstract: Appropriate reliance on AI advice has become a central research theme in human-AI collaboration.
arXiv AI
Jul 17
Towards a Unified Multidimensional Explainability Metric: Evaluating Trustworthiness in AI Models
arXiv:2607. 14315v1 Announce Type: cross Abstract: In this paper, we present a comprehensive framework for assessing the explainability of various XAI methods, such as LIME and SHAP, across multiple datasets and machine learning models, with the ultimate goal of creating a unified multidimensional explainability score.
OpenAI Blog
Sep 8, 2021
TruthfulQA: Measuring how models mimic human falsehoods
arXiv AI
Jun 11
The Impossibility of Eliciting Latent Knowledge
arXiv:2606. 12268v1 Announce Type: new Abstract: Advanced AI systems have extensive knowledge of their environments; in fact, their knowledge may (far) exceed that of their developers or users.
arXiv AI
5d ago
Position: Reasoning is a Learnable Rule-Based Process
arXiv:2608. 12325v1 Announce Type: new Abstract: Autonomous reasoning is among the most scientifically and economically motivating topics in AI today.
arXiv AI
Jul 17
"Trust Junk" Leads to Unjustified Support for Highly Discriminatory Predictive Models
arXiv:2607. 14152v1 Announce Type: cross Abstract: The persuasive power of data visualizations can go awry: for instance, in an explainable AI (XAI) context, visualizations can produce over-trust of predictive models.
arXiv Machine Learning
1d ago
Limits to scalable evaluation at the frontier: LLM as Judge won't beat twice the data
arXiv:2410. 13341v4 Announce Type: replace Abstract: High quality annotations are increasingly a bottleneck in the explosively growing machine learning ecosystem.