Goldsmith: Gold-Loss-Guided Definition Optimization with an Agentic Annotation Harness
Read the original on arXiv Computation and Language →Goldsmith is an agentic pipeline that transforms a small set of expert-annotated calibration examples into a reusable structured annotation definition treated as a trainable textual object. Candidate definitions are evaluated on the same gold examples using an executable structured loss, and a large language model editor revises high-loss failures only when loss decreases. The approach outperforms direct rewriting, OPRO, APE, and PromptBreeder in prompt-optimization tests and enhances downstream annotation across typed span, pair-level relation, and fixed-trigger event-argument tasks when combined with retrieval, score-based routing, and human review.
Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.