← Back to all news
arXiv Computation and Language By Obinna I. Ekekezie

Can LLMs in Draft-Verify-Revise Pipelines Resolve Deictic Ambiguity?

Read the original on arXiv Computation and Language →

The study investigates how deictic ambiguity—specifically the shifting reference of expressions like "previous"—affects Draft‑Verify‑Revise pipelines that use multiple large language models (LLMs). Using a synthetic dataset of 10 base examples and 21 reasoning‑effort configurations, six LLMs were evaluated for their ability to correctly resolve the ambiguous expression across the draft, verify, and revise stages. Results show wide variance in balanced accuracy, with GPT‑5.2 improving from 0.156 to 0.942 with increased reasoning effort, while Gemini 3 Pro consistently achieved high accuracy above 0.94 even at low reasoning effort, and meta‑evaluators often relied on surface cues when making errors.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

Related stories

arXiv AI
Aug 11

Reproducing and Stress-Testing Two Approaches to LLM Reasoning Reliability: Test-Time Probability Aggregation and Logic-Representation Editing

arXiv:2608. 08514v1 Announce Type: new Abstract: We independently reproduce two recent methods for making large language model (LLM) reasoning more reliable, and stress-test them across domains and models (RPC across four new task domains with Qwen3-8B, LCF across four 7-8B models).

By Minhan Cho, Jimin Kweon
llmsdata-engineering
More like this →
arXiv AI
Jul 22

Prompt Design at Scale: How Format, Instruction Count, and Context Length Shape Instruction Adherence and Hallucination in Large Language Models

arXiv:2607. 19257v1 Announce Type: cross Abstract: Practitioners make three prompt-design decisions with almost no controlled evidence behind them: how to format instructions and context (markdown, plain text, prose, or tabular), how many simultaneous instructions a system prompt can carry before compliance degrades, and how much context a model can hold before recall and honesty degrade.

By Netanel Eliav
llmssafety
More like this →
Hugging Face Trending Papers
Jul 21

Prompt Design at Scale: How Format, Instruction Count, and Context Length Shape Instruction Adherence and Hallucination in Large Language Models

Practitioners make three prompt-design decisions with almost no controlled evidence behind them: how to format instructions and context (markdown, plain text, prose, or tabular), how many simultaneous instructions a system prompt can carry before compliance degrades, and how much context a model can hold before recall and honesty degrade. We report two controlled experiments crossing all three factors on one held, contamination-free synthetic corpus (the "Book of Veyra," 8,780 uniquely-named entities, deterministically regenerable from a fixed seed), evaluated across five models.

llmssafety
More like this →
arXiv Computation and Language
3d ago

Draft-OPD: On-Policy Distillation for Speculative Draft Models

Draft-OPD introduces an on‑policy distillation method for speculative draft models, addressing the mismatch between supervised fine‑tuning and inference by letting the target model supervise the drafter on draft‑induced states. The approach uses target‑assisted rollouts for stable continuations and replays drafting from error positions exposed during verification, enabling the drafter to learn from both accepted and rejected proposals. Experiments demonstrate that Draft‑OPD achieves more than five‑fold lossless acceleration across diverse tasks, outperforming prior draft models such as EAGLE‑3 and DFlash by 23 % and 13 % respectively.

By Haodi Lei, Yafu Li, Haoran Zhang, Shunkai Zhang, Qianjia Cheng, Xiaoye Qu, Ganqu Cui, Bowen Zhou, Ning Ding, Yun Luo, Yu Cheng
llmsfine-tuningefficiency
More like this →