ScienceJudge introduces Pattern Perturbation Augmentation (PPA) to generate hard negative examples for sentence‑level scientific claim verification by modeling reporting errors as faithfulness‑breaking transformations. PPA applies these patterns to supported claims, creating controlled hard negatives and expanding coverage through pattern‑level, operation‑level, and cross‑domain augmentation. The resulting ScienceJudge system, which ensembles PPA‑trained vision‑language predictors with instruction‑guided LLM predictors, achieves state‑of‑the‑art performance, improving the average score from 0.8124 to 0.8692 and ranking first on the NLPCC 2026 Task 10 Track 1 leaderboard.
arXiv:2609.22774v1 Announce Type: new Abstract: This paper presents our system for Track 2 of the NLPCC 2026 Shared Task 10 on citation-level faithfulness in AI-assisted scientific reporting. Given a...
This paper presents an overview of \emph{SciClaimEval}, a pilot task at NTCIR-19 on verifying scientific claims against the tables and figures of the papers in which they appear. Our dataset pairs aut...
arXiv:2611.06090v1 Announce Type: new Abstract: This paper presents an overview of \emph{SciClaimEval}, a pilot task at NTCIR-19 on verifying scientific claims against the tables and figures of the p...
The paper investigates how stylistic changes introduced by large language models (LLMs) affect multimodal claim verification, a task that determines whether a textual claim is supported by given evidence. Two rewriting strategies are used: natural rewriting, mimicking typical academic polishing, and controlled injection, adding a single LLM-associated word. Across 11 open‑weight models (2B–38B parameters) from five VLM families, the study finds that most models remain robust to these modifications, showing no significant accuracy drop, though consistent probability shifts—especially under hedging conditions—are observed.
arXiv:2608. 09209v1 Announce Type: cross Abstract: Neural language models trained on large crowdsourced corpora frequently exploit spurious surface patterns tied to target labels without true linguistic or causal relevance, boosting benchmark performance while failing on adversarial or out-of-distribution inputs.
arXiv:2606. 02837v1 Announce Type: cross Abstract: Accurate translation from Natural Language to First-Order Logic (NL-to-FOL) underpins neurosymbolic AI systems and Natural Language Inference (NLI), making the quality of NL-to-FOL benchmarks essential -- yet these datasets have never been rigorously audited.
arXiv:2610.16441v1 Announce Type: cross Abstract: Current image-text evaluation often relies on a global score, which gives limited insight into which semantic details are supported by the image. Exp...
The Writerslogic team participated in the CLEF 2026 SimpleText shared task, tackling both text simplification (Task 1) and complexity spotting (Task 2). For simplification, they built a multi‑candidate pipeline with GPT‑4o‑mini, selecting the best candidate via a reference‑free heuristic, and their Claude Sonnet 4 submission achieved a SARI of 47.43 and BLEU of 14.21, ranking third overall on the Task 1 leaderboard. For complexity spotting, they fine‑tuned a DeBERTa‑v3‑large NLI model on 350 K labeled pairs, achieving a macro F1 of 0.8081 (0.8085 in an ensemble) on binary over‑generation identification and 0.804 accuracy on multi‑class error classification, placing them second among unique teams.
The study evaluates hybrid retrieval, NLI, and open‑source LLMs for scientific claim verification. Using a benchmark of 2,035 claims, the authors compare a DeBERTa‑based NLI baseline with Llama 3.1, Qwen3, and Mistral LLMs, finding that Qwen3 achieves the highest Macro‑F1 score. A conservative fusion rule that labels a claim as Support or Contradict only when both models agree actually degrades performance, and error analysis reveals that disagreements often overturn correct LLM predictions.
arXiv:2607. 16131v1 Announce Type: cross Abstract: Multimodal Scientific Claim Verification (MSCV) requires models to verify scientific claims using visually grounded evidence from papers, including figures, tables, charts, and textual context.
Neural language models trained on large crowdsourced corpora frequently exploit spurious surface patterns tied to target labels without true linguistic or causal relevance, boosting benchmark performance while failing on adversarial or out-of-distribution inputs. Existing approaches either require manual specification of the feature vocabulary or automate discovery only partially, leaving the gap between dataset-level correlation and model-level exploitation unaddressed.
arXiv:2610.32128v1 Announce Type: new Abstract: RAG and retrieval-augmented few-shot ICL can struggle when decision-relevant evidence is buried in long, noisy context. Retrieved passages and demonstr...