arXiv AI By Sujato Dutta, Prithviraj Patil, Chandrajit Bajaj

Knowledge After Collapse: Context and Wording Shape Factual Accessibility in Language Models

Read the original on arXiv AI →

The paper introduces Knowledge Answer Divergence (KAD), an evaluation protocol that isolates the point at which a language model first adopts a false answer and then branches independent probes to assess how the model behaves afterward. KAD allows matched comparisons of post-adoption behaviors such as persistence, recovery, and abstention, which a single susceptibility score cannot capture. Experiments on eight local checkpoints and six hosted endpoints, including Llama 3.2 Instruct 3B and Qwen 3.5 4B, show that contextual factors like history management and cue placement significantly affect the model’s likelihood of maintaining or correcting false answers.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI
Sep 17

AfriSyCo: Measuring Assertive Framing, Verification, and Wording Sensitivity Around African-Language Content

AfriSyCo investigates how different framing and verification strategies affect the accuracy of language models on African‑language factual content. The study uses a cross‑language factorial design with native‑language follow‑ups and English framing, analyzing 1,415 observations from 100 source questions across seven checkpoints and six languages. Results show that assertive framing boosts target selection by up to 30.4 points, while verification reduces it by 17.4 points, with strong interactions and large variability depending on wording and checkpoint.

By David Ababio Awuni, Rose-Mary Owusuaa Mensah Gyening, Elvis Gyasi Owusu
arXiv AI
Nov 6

Signals, Evidence, and Context: User Scrutiny in Real-World LLM Conversations

The study examines how features of large language model (LLM) responses influence user scrutiny—accepting, checking, or challenging—across 489,283 assistant-user pairs from ShareChat and WildChat. It finds that broad attribution language appears in 31.9% of qualifying responses, while explicit URLs are present in only 1.33%, indicating a gap between claimed grounding and inspectable evidence. Various response cues such as attribution, uncertainty, and reasoning traces are linked to both checking and challenging, with uncertainty slightly favoring checking overall and differing effects observed in Technical Help versus Seeking Information contexts.

By Ahana Biswas, Yu-Ru Lin
arXiv AI
Oct 16

Learning When to Withdraw: Curriculum Coverage and Rule-Specific Failures in Fine-Tuned Language Agents

The paper investigates how fine‑tuned language agents can selectively withdraw actions when evidence changes or authority is revoked. Using a synthetic NAQD‑Env environment, the authors compare an original curriculum with an enriched one across multiple system‑prompt framings and evaluate four LoRA checkpoints of Qwen2.5‑3B‑Instruct against instruction‑tuned baselines. Results show that fine‑tuned checkpoints achieve higher decision agreement and withdrawal recall, reduce incorrect withdrawals, and narrow prohibited combinations, though they also omit valid event reports and exhibit mixed preservation of authorized actions.

By Mohamed Abouzahra
arXiv Computation and Language
Sep 16

EviScope: Paired Counterfactual Evidence Diagnostics for Faithful and Efficient Grounded Language Models

EviScope is a new paired counterfactual benchmark that evaluates grounded language models by fixing the question while manipulating evidence—adding, removing, distracting, or contradicting it. The v1.1 dataset includes 40 four‑condition quartets with repaired counterfactual claims and span‑level support labels for automated assessment. Experiments on Qwen2.5‑7B, Llama 3.1 8B, and Gemini 3.5 Flash show that paired metrics reveal grounding behaviors hidden by simple answer accuracy, such as unsupported answers, conflict blindness, and incorrect non‑answer actions.

By Suryadeep Singh Deswal