arXiv AI
Reach Into The CHOIR: Free-List Elicitation Uncovers Distinct Model Voices in LLM Ensembles
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Sep 1
The Unsampled Truth: Quantifying Prompt Artifacts in LM Psychometrics
The study investigates how different prompt components affect language model responses in psychometric tests. By crossing five distinct baseline personas with five variants of each prompt element—persona wording, task instruction, item wording, and option symbol—the authors measure response shifts using the 1‑Wasserstein distance. Their analysis of 13 small open‑weight language models on the Big Five Inventory and Short Dark Triad reveals that task instruction and option symbol changes often cause more variation than paraphrasing the persona or item, with prompt artifacts explaining over 50% of the variation for many items.
arXiv Machine Learning
Aug 14
A Probe Direction Is a Property of Its Prompt
arXiv:2608. 13329v1 Announce Type: new Abstract: A model that behaves differently when it senses it is being tested would undermine the evaluations we rely on, so recent work has sought to read that sense directly from a model's activations.
Hugging Face Trending Papers
Jun 17
The Wrong Kind of Right: Quantifying and Localizing Misfired Alignment in LLMs
Warning: This paper studies stereotypes and biases, and contains potentially disturbing examples, used for illustration purposes only. Our findings should not be interpreted as an argument against alignment.
arXiv AI
Jul 29
CogArena: A Multimethod Evaluation of Cognitive Ability Structure in Large Language Models
arXiv:2607. 24999v1 Announce Type: cross Abstract: LLM cognitive scores are increasingly summarized as per-ability profiles whose dimensions should converge across tasks, respond selectively to matched interventions, and generalize beyond the models used to define them.
arXiv Machine Learning
Sep 22
Evaluation Awareness Shifts from Format to Context with Model Scale
arXiv:2609.22119v1 Announce Type: cross Abstract: Evaluation awareness poses an unprecedented threat to model evaluation, but the mechanisms by which models detect it remain unknown. This study focus...
arXiv Computation and Language
Sep 1
How You Ask Shapes What You Get: A Theory-Seeded Measurement of Articulation in Advice-Seeking LLM Conversations
The paper investigates how the way users phrase advice‑seeking requests—termed articulation—creates stable, measurable patterns distinct from the topics of the requests. By analyzing 16,447 prompts from public chat corpora, the authors identify a small set of latent articulation factors that consistently appear across datasets and splits. One key finding is a long‑form, information‑poor style that leads language models to give shorter, vaguer answers without seeking clarification, a pattern that persists across topics and prompt lengths.