WildSEEK: Evaluating Language Models for Information-Seeking

WildSEEK is a new dataset of 3,000 real user information‑seeking queries, manually annotated for risk‑sensitive domains and whether the query is factoid or analytical. The accompanying evaluation framework tests LLM responses against four failure criteria—sycophantic behavior, overreliance, a default US‑centric perspective, and poor handling of vulnerable populations—finding higher failure rates for analytical queries. The authors also train classifiers on WildSEEK to analyze over 1.8 million realistic queries, revealing that more than a third are high‑risk and often analytical.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

• llms
• safety