arXiv AI By Nils Schwager, Christoph Hau, Simon M\"unker, Achim Rettinger

The Unsampled Truth: Quantifying Prompt Artifacts in LM Psychometrics

Read the original on arXiv AI →

The study investigates how different prompt components affect language model responses in psychometric tests. By crossing five distinct baseline personas with five variants of each prompt element—persona wording, task instruction, item wording, and option symbol—the authors measure response shifts using the 1‑Wasserstein distance. Their analysis of 13 small open‑weight language models on the Big Five Inventory and Short Dark Triad reveals that task instruction and option symbol changes often cause more variation than paraphrasing the persona or item, with prompt artifacts explaining over 50% of the variation for many items.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI
Aug 28

Investigating the Influence of Prompt and Response Languages on LLM Content Generation

The paper investigates how the language of prompts and responses affects large language model (LLM) outputs. Using five models and 68 non‑translation questions, the authors compare English‑to‑English, English‑to‑Norwegian, Norwegian‑to‑Norwegian, and Norwegian‑to‑English conditions, yielding 1,348 responses after filtering. They find that prompt language strongly influences response length—Norwegian prompts shorten English outputs by ~37 % and English prompts shorten Norwegian outputs by ~41 %—while semantic similarity remains high across conditions.

By Thi Thanh Nhan Nguyen, Mai Khoi Tieu, Michael A. Riegler, P{\aa}l Halvorsen, Thu Nguyen