OpenAI Blog
TruthfulQA: Measuring how models mimic human falsehoods
Read the original on OpenAI Blog →The Flow has not summarised this story yet — read it at OpenAI Blog.
Related stories
arXiv AI
Jun 26
ConflictScore: Identifying and Measuring How Language Models Handle Conflicting Evidence
arXiv:2606. 26437v1 Announce Type: cross Abstract: Existing metrics for factuality and faithfulness evaluate whether an answer is supported or contradicted by its grounding documents, but they fail to capture when both supporting and contradicting evidence coexist.
arXiv AI
Jun 17
PseudoBench: Measuring How Agentic Auto-Research Fuels Pseudoscience
arXiv:2606. 18060v1 Announce Type: new Abstract: As Large Language Model based agents enter autonomous scientific research, their ability to resist pseudoscience becomes increasingly important.
arXiv AI
Aug 12
Graphical Models of False Information and Fact Checking Ecosystems
arXiv:2208. 11582v2 Announce Type: replace-cross Abstract: The wide spread of false information online, including misinformation and disinformation, has become a major problem for our highly digitised and globalised society.
arXiv AI
Jun 12
"Did you lie?" Evaluating Lie Detectors across Model Scale and Belief-Verified Model Organisms
arXiv:2606. 12618v1 Announce Type: new Abstract: Robust lie detectors for language models could enable powerful techniques for auditing, monitoring, and post-hoc investigation of model behaviour, but evaluating them requires testbeds where models verifiably believe the opposite of what they say.
arXiv AI
5d ago
Falsehood and Impossibility Are Different Directions in an AI's Representation of Language
arXiv:2608. 12852v1 Announce Type: cross Abstract: Language can describe states of affairs that are false and states of affairs that could not be the case at all.
arXiv AI
Jul 17
"Trust Junk" Leads to Unjustified Support for Highly Discriminatory Predictive Models
arXiv:2607. 14152v1 Announce Type: cross Abstract: The persuasive power of data visualizations can go awry: for instance, in an explainable AI (XAI) context, visualizations can produce over-trust of predictive models.