arXiv AI By Justin Bronder (Corabo Inc.)

Instrument Effects in Language-Model Honesty Evaluation: An Auditable Single-System Demonstration

Read the original on arXiv AI →

arXiv:2607. 14399v1 Announce Type: new Abstract: Evaluations of language-model honesty read the model's verdicts as evidence about the model.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv AI.

arXiv AI
2d ago

Inducing Reward-Free Judging Rubrics that Reduce Over-Crediting in Agent Evaluation

arXiv:2608. 13564v1 Announce Type: new Abstract: Evaluating language-model agents at scale increasingly relies on a second language model as an automatic judge, because the gold signal, an executable environment reward, is expensive, slow, or unavailable at deployment time.

By Darragh Quinn, David Dylan, Roisin Healy, Fionn Carroll, Maeve Donnelly, Cormac Sheehan