arXiv:2608.23244v1 Announce Type: cross
Abstract: Large language models (LLMs) often produce fluent but incorrect answers with unwarranted confidence. A central limitation is that standard LLMs repre...
By Shireen Kudukkil Manchingal, Sofiia Nikolenko, Fabio Cuzzolin
The paper investigates hallucination detection in black‑box large language models by leveraging two accessible signals: semantic entropy, which captures disagreement among sampled response meanings, and token‑level uncertainty derived from log‑probabilities. It introduces a TopK aggregation technique, a hybrid CoCoA method combining uncertainty with semantic dissimilarity, and two supervised approaches—Gated and Stacked—that integrate token and semantic features. Across seven benchmarks and four language models, the supervised Stacked method performs best in many cases, while TopK and CoCoA remain competitive without labeled data, though all methods require careful threshold calibration.
By Urja Pawar, Rajitha Ramanayake, Owen O'Neill, Nabeel Kemal, Abhishek Mandal, Houssem Chatbri, Christopher Martin
arXiv:2606. 07822v1 Announce Type: cross Abstract: As language models improve and become increasingly deployed to solve a variety of tasks, trustworthiness becomes essential.
By Nishant Subramani, Palash Goyal, Yiwen Song, Mani Malek, Yuan Xue, Tomas Pfister, Hamid Palangi
arXiv:2605. 27752v2 Announce Type: replace Abstract: LLM confidence calibration is often evaluated by comparing two signals: token-probability scores and verbalized confidence.
By Hankyeol Kim, Pilsung Kang
arXiv:2607. 19361v1 Announce Type: cross Abstract: Most safety guardrails for large language models (LLMs) evaluate each prompt-response pair in isolation, which misses failures that arise only over a dialogue as benign turns compose into harm.
By Sanjay Mishra, Divya Chukkapalli, Ganesh R. Naik
The paper introduces SEAV, a verification‑centric framework for evaluating jailbreak attempts against large language models. SEAV decomposes responses into ordered steps and checks both validity and correctness using LLM‑as‑a‑judge and retrieval‑grounded verification. The method reduces false positives by 14.9 percentage points on a strategic‑dishonesty diagnostic and reclassifies 22.1–51.0% of previously successful jailbreaks as invalid across multiple benchmarks.
By Qilong Wu, Sahil Wadhwa, Pranab Mohanty, Giri Iyengar, Varun Chandrasekaran