arXiv AI
Sep 24

When Context Misleads: In-context Learning with Jurisdiction in Large Language Models

The paper "When Context Misleads: In-context Learning with Jurisdiction in Large Language Models" introduces FakeContextBench, a benchmark of pseudoscientific claims across seven domains to test large language models’ ability to judge whether contextual information should dictate the answer. Experiments show that both commercial and open‑source models struggle with context authority and that standard ICL fine‑tuning can worsen susceptibility to misleading context. The authors propose Jurisdiction In‑Context Learning (J‑ICL), a post‑training framework that adds context validation to the objective, which improves benchmark scores and reality accuracy by significant margins over baseline models.

By Pei-lin Li, Qingle Liu, Junyang Feng, Siyu Li, Sunqi Fan, Xin-Sheng Chen, Shuojin Yang