arXiv AI By Luca Giordano, Simon Razniewski

Beyond Questions: Evaluating LLM's Knowledge Expression

Read the original on arXiv AI →

arXiv:2605. 26937v2 Announce Type: replace-cross Abstract: Parametric knowledge in large language models (LLMs) is a cornerstone of their success, yet remains poorly understood.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI
Aug 25

What Models Know, How Well They Know It: Knowledge-Weighted Fine-Tuning for Learning When to Say "I Don't Know"

The paper introduces Knowledge-Weighted Fine‑Tuning, a method that estimates an instance‑level knowledge score through multi‑sampled inference and uses it to scale the learning signal. This approach encourages large language models to explicitly say "I don't know" on out‑of‑scope queries while preserving accuracy on known questions. The authors also propose new evaluation metrics for uncertainty, demonstrating that better discrimination between known and unknown instances improves overall performance.

By Joosung Lee, Hwiyeol Jo, Donghyeon Ko, Kyubyung Chae, Cheonbok Park, Jeonghoon Kim
arXiv AI
Jun 10

RankLLM: Weighted Ranking of LLMs by Quantifying Question Difficulty

arXiv:2602. 12424v2 Announce Type: replace-cross Abstract: Benchmarks establish a standardized evaluation framework to systematically assess the performance of large language models (LLMs), facilitating objective comparisons and driving advancements in the field.

By Ziqian Zhang, Xingjian Hu, Yue Huang, Kai Zhang, Ruoxi Chen, Yixin Liu, Qingsong Wen, Kaidi Xu, Xiangliang Zhang, Neil Zhenqiang Gong, Lichao Sun
arXiv AI
Jun 10

LakeQA: An Exploratory QA Benchmark over a Million-Scale Data Lake

arXiv:2606. 10460v1 Announce Type: cross Abstract: Recent large language models (LLMs) have shown rapid progress in reading-based question answering (QA), where evidence is explicitly provided or can be trivially retrieved.

By Haonan Wang, Jiaxiang Liu, Yurong Liu, Austin Senna Wijaya, Tianle Zhou, Eden Wu, Yijia Chen, Wanting You, Reya Vir, Daniela Pinto, Grace Fan, Yusen Zhang, Juliana Freire, Eugene Wu