arXiv AI
Jun 24
CAVEWOMAN: How Large Language Models Behave Under Linguistic Input and Output Compression
arXiv:2606. 24083v1 Announce Type: cross Abstract: "Talk short.
Structured Output Collapses Answer Diversity Across 44 Language Models
arXiv:2607. 18476v1 Announce Type: cross Abstract: When a language model must choose one answer from a large space of equally valid options, a format clause -- "Reply with JSON only" -- changes which answer it chooses.
Related stories
arXiv AI
Jul 15
The One-Word Census: Answer-Choice Conformity Across 44 Language Models
arXiv:2607. 12796v1 Announce Type: cross Abstract: When a language model must pick one answer from a large space of equally valid options, which does it pick -- and how often is it the same answer every other model picks?
arXiv Computation and Language
6d ago
Where a Model Sends Its Own Repeated Token
arXiv:2609. 31181v1 Announce Type: new Abstract: Black-box model identification works by scoring a model's response to natural-language prompts.
arXiv AI
Jul 24
PhantomFill: When the Form Demands an Answer, Language Models Invent One
arXiv:2607. 20492v1 Announce Type: cross Abstract: Language models in production do not write prose.
arXiv AI
Jul 22
Prompt Design at Scale: How Format, Instruction Count, and Context Length Shape Instruction Adherence and Hallucination in Large Language Models
arXiv:2607. 19257v1 Announce Type: cross Abstract: Practitioners make three prompt-design decisions with almost no controlled evidence behind them: how to format instructions and context (markdown, plain text, prose, or tabular), how many simultaneous instructions a system prompt can carry before compliance degrades, and how much context a model can hold before recall and honesty degrade.
Hugging Face Trending Papers
Jul 14
The One-Word Census: Answer-Choice Conformity Across 44 Language Models
When a language model must pick one answer from a large space of equally valid options, which does it pick -- and how often is it the same answer every other model picks? Asked to "pick a word -- any word," 44 models chose "serendipity" 41% of the time.
arXiv AI
Jun 9
Capacity, Not Format: Rethinking Structured Reasoning Failures
arXiv:2606. 09410v1 Announce Type: new Abstract: Prior work treats structured output as a reasoning tax, but this framing is incomplete: the cost of formatting depends strongly on a model's spare capacity.
arXiv AI
Jul 7
Detecting Hallucinations in Retrieval-Augmented Generation through Grounding-Aware Sensitivity by Perturbation (GASP)
arXiv:2607. 04223v1 Announce Type: cross Abstract: Retrieval-augmented generation (RAG) reduces but does not eliminate hallucination, and existing detectors return a single answer-level score that does not indicate which sentence is unsupported, or why.
arXiv Computation and Language
3d ago
Listening to the Wise Few: Query-Key Alignment Unlocks Latent Correct Answers in Large Language Models
arXiv:2410.02343v2 Announce Type: replace Abstract: Large language models (LLMs) routinely fail to output the correct option in multiple-choice question answering (MCQA) while encoding the answer int...
arXiv AI
Jul 7
They Infer What You Meant: Models Represent Communicative Intent More Reliably Than They Act On It
arXiv:2607. 03598v1 Announce Type: cross Abstract: When a person shares something with a language model, the model often answers the surface of the message rather than what the sender was doing by sending it: share a finished project and it critiques the code; share a raw late-night line and it runs a wellness check.
arXiv Computation and Language
Sep 22
this-that-model-1.0: A typed decision model that decides in 30 ms, for a millionth of a cent
arXiv:2609.23886v1 Announce Type: new Abstract: Software delegates more of its branches to models every year: which queue a ticket enters, whether a command is safe to run, whether a claim clears wit...
arXiv AI
2d ago
Measuring the Microtask Eligibility Gap: When Is an Off-the-Shelf SLM Enough for an Agent Harness?
The paper introduces a benchmark for evaluating whether off‑the‑shelf small language models (SLMs) can reliably perform microtasks that support a large language model (LLM) planner, such as auto‑approving shell commands, writing memory, selecting tools, and ranking past turns. Using fixed prompts and confidence‑interval‑aware eligibility thresholds, the authors test several Qwen3 models (0.6/1.7/4/8 B) in FP16 with no tuning and find that none of the 16 configurations meet the eligibility criteria. Quantization to 4‑bit precision further degrades performance, with the eligibility gap tracking model size rather than precision, and the issue persists across different models (e.g., Llama‑3.x) and prompt variations.