arXiv AI September 17, 2026 By David Ababio Awuni, Rose-Mary Owusuaa Mensah Gyening, Elvis Gyasi Owusu

AfriSyCo: Measuring Assertive Framing, Verification, and Wording Sensitivity Around African-Language Content

AfriSyCo investigates how different framing and verification strategies affect the accuracy of language models on African‑language factual content. The study uses a cross‑language factorial design with native‑language follow‑ups and English framing, analyzing 1,415 observations from 100 source questions across seven checkpoints and six languages. Results show that assertive framing boosts target selection by up to 30.4 points, while verification reduces it by 17.4 points, with strong interactions and large variability depending on wording and checkpoint.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI

Aug 25

Evaluation Awareness in Language Models: Representation, Verbalization, and Control

arXiv:2608.21766v1 Announce Type: cross Abstract: Both capability and safety benchmarks rest upon the assumption that the behavior of language models undergoing a test is informative about their beha...

By Farzaneh Heidari, Amin Memarian, Guillaume Rabusseau

llms fine-tuning benchmarks safety

arXiv Computation and Language

2d ago

EviScope: Paired Counterfactual Evidence Diagnostics for Faithful and Efficient Grounded Language Models

EviScope is a new paired counterfactual benchmark that evaluates grounded language models by fixing the question while manipulating evidence—adding, removing, distracting, or contradicting it. The v1.1 dataset includes 40 four‑condition quartets with repaired counterfactual claims and span‑level support labels for automated assessment. Experiments on Qwen2.5‑7B, Llama 3.1 8B, and Gemini 3.5 Flash show that paired metrics reveal grounding behaviors hidden by simple answer accuracy, such as unsupported answers, conflict blindness, and incorrect non‑answer actions.

By Suryadeep Singh Deswal

llms rag benchmarks

arXiv AI

Jun 12

Prefill Awareness in Large Language Models

arXiv:2606. 12747v1 Announce Type: new Abstract: Safety-relevant studies of language models, including alignment and jailbreaking evaluations and AI control protocols, often rely on prefilling model outputs.

By Andy Wang, Parv Mahajan, David Demitri Africa, Alexandra Souly, Jordan Taylor, Robert Kirk

llms agents benchmarks safety

arXiv Machine Learning

Jul 30

Same Evidence, Different Target: Decoding How Diagnostic Evidence Bears on Causal Questions from Language-Model States

arXiv:2607. 26929v1 Announce Type: cross Abstract: The same diagnostic result can support or challenge one causal claim yet fail to address another when the claims concern different populations, outcomes, estimands, pathways, or identifying assumptions.

By Weiyi Kong, Zhuoran Li

llms benchmarks

arXiv Machine Learning

Jul 16

Graded Entity-Familiarity Readouts in Language Models: Polish Adaptation, Cross-Language Robustness, and Refusal Steering

arXiv:2607. 13568v1 Announce Type: cross Abstract: Can a language model estimate its familiarity with an entity before generating an answer?

By Grzegorz Brzezinka

llms

Hugging Face Trending Papers

3d ago

EviScope: Paired Counterfactual Evidence Diagnostics for Faithful and Efficient Grounded Language Models

Grounded language-model systems are often evaluated by final answer accuracy, yet a correct answer can be unsupported, drawn from the wrong source, or produced when evidence is insufficient or contrad...

llms rag benchmarks

One email a morning, machine-written

Related stories

Evaluation Awareness in Language Models: Representation, Verbalization, and Control

EviScope: Paired Counterfactual Evidence Diagnostics for Faithful and Efficient Grounded Language Models

Prefill Awareness in Large Language Models

Same Evidence, Different Target: Decoding How Diagnostic Evidence Bears on Causal Questions from Language-Model States

Graded Entity-Familiarity Readouts in Language Models: Polish Adaptation, Cross-Language Robustness, and Refusal Steering

EviScope: Paired Counterfactual Evidence Diagnostics for Faithful and Efficient Grounded Language Models