arXiv Computation and Language October 19, 2026 By Shamim Hasan, Samiya Ahmed Chowdhury

Bengali Hallucination Detection with Retrieval and Programs: A System Study

Read the original on arXiv Computation and Language →

The paper presents a Bengali hallucination detection system that uses response‑conditioned query rewriting, hybrid retrieval, and task‑specific language‑model judging. It employs six retrieval routes and a router that selects between a 27B textual judge and a 14B agent capable of reasoning or executing Python. The authors evaluate the system on the Olikbochon challenge, reporting a private macro‑F1 of 0.956 and detailing runtime, execution traces, and corpus overlap findings.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

arXiv AI

Jun 2

BenHalluEval: A Multi-Task Hallucination Evaluation Framework for Large Language Models on Bengali

arXiv:2605. 31483v1 Announce Type: cross Abstract: Despite Bengali being the sixth most spoken language in the world, no prior work has systematically evaluated hallucination in large language models (LLMs) for Bengali.

By Shefayat E Shams Adib, Ahmed Alfey Sani, Ekramul Alam Esham, Ajwad Abrar, Ishmam Tashdeed, Md Taukir Azam Chowdhury

llms nlp benchmarks safety

arXiv AI

Jun 30

How Far Can You Get Without a GPU? A Systematic Benchmark of Lightweight Hallucination Detection Across Question Answering, Dialogue, and Summarisation

arXiv:2606. 29809v1 Announce Type: cross Abstract: Hallucination detection has become a pressing requirement for trustworthy AI deployment at scale.

By Kriti Faujdar, Smit Kadvani

nlp benchmarks safety

arXiv Computation and Language

Oct 5

Writerslogic at the CLEF 2026 SimpleText Track: Multi-Candidate LLM Simplification and Stacked Complexity Spotting

The Writerslogic team participated in the CLEF 2026 SimpleText shared task, tackling both text simplification (Task 1) and complexity spotting (Task 2). For simplification, they built a multi‑candidate pipeline with GPT‑4o‑mini, selecting the best candidate via a reference‑free heuristic, and their Claude Sonnet 4 submission achieved a SARI of 47.43 and BLEU of 14.21, ranking third overall on the Task 1 leaderboard. For complexity spotting, they fine‑tuned a DeBERTa‑v3‑large NLI model on 350 K labeled pairs, achieving a macro F1 of 0.8081 (0.8085 in an ensemble) on binary over‑generation identification and 0.804 accuracy on multi‑class error classification, placing them second among unique teams.

By David L. Condrey

llms fine-tuning benchmarks safety

arXiv AI

Jun 26

Ask, Don't Judge: Binary Questions for Interpretable LLM Evaluation and Self-Improvement

arXiv:2606. 27226v1 Announce Type: new Abstract: Evaluating LLM outputs remains a major bottleneck in NLP: human evaluation is expensive and slow, lexical metrics correlate poorly with human judgments on open-ended generation, and holistic LLM judges often produce opaque scores that are hard to debug.

By Sangwoo Cho, Kushal Chawla, Pengshan Cai, Zefang Liu, Chenyang Zhu, Shi-Xiong Zhang, Sambit Sahu

llms nlp benchmarks

arXiv Computation and Language

Sep 14

Tasks over Application Manuals: Revealing Gaps in Long-Horizon Procedural Reasoning for Language Models

The paper introduces Tasks over Application Manuals (TAM), a benchmark designed to test long‑horizon procedural reasoning in large language models. TAM uses real‑world tasks from ICD‑10‑CM clinical coding and U.S. federal sentencing, requiring models to follow extensive, rule‑based manuals and perform interdependent steps to produce exact answers. Experiments with GPT‑5 and various prompting strategies show very low exact‑match accuracy—1% for coding and 15.5% for sentencing—highlighting a gap between current benchmarks and the ability to reliably follow complex procedures.

By Utkarsh Soni, Syed Shariyar Murtaza, Yifan Nie, Sachin Chandrasekhar, Eugene Wen

llms rag agents benchmarks

arXiv AI

2d ago

Adaptive Retrieval, Selective Answers: Evaluating Conformal Factuality in Agentic RAG

arXiv:2610.34872v1 Announce Type: cross Abstract: Retrieval-augmented generation (RAG) can improve factual grounding without guaranteeing that every generated claim is supported. Previous work has st...

By Muhammad Aimal Rehman, Chi-Kuang Yeh

llms rag agents

One email a morning, machine-written

Related stories

BenHalluEval: A Multi-Task Hallucination Evaluation Framework for Large Language Models on Bengali

How Far Can You Get Without a GPU? A Systematic Benchmark of Lightweight Hallucination Detection Across Question Answering, Dialogue, and Summarisation

Writerslogic at the CLEF 2026 SimpleText Track: Multi-Candidate LLM Simplification and Stacked Complexity Spotting

Ask, Don't Judge: Binary Questions for Interpretable LLM Evaluation and Self-Improvement

Tasks over Application Manuals: Revealing Gaps in Long-Horizon Procedural Reasoning for Language Models

Adaptive Retrieval, Selective Answers: Evaluating Conformal Factuality in Agentic RAG