arXiv Machine Learning
Evaluation of Contextual Understanding in Large Language Models
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv AI
Sep 28
Do LLMs Understand Context? A Knowledge Graph-Based Evaluation Framework
The paper introduces a knowledge‑graph‑based evaluation framework, S3KG, to assess whether large language models truly understand context in question answering tasks. S3KG combines structural and semantic signals into a single similarity score and is paired with a diagnostic analysis that pinpoints reasoning errors at the triplet level. Across nine benchmarks, the method outperforms existing baselines, achieving up to +7.6 F1 points and an AUROC of 0.973.
arXiv AI
Jun 6
Improving Answer Extraction in Context-based Question Answering Systems Using LLMs
arXiv:2606. 06197v1 Announce Type: cross Abstract: Question answering (QA) systems have achieved notable progress with the advent of large language models (LLMs).
arXiv AI
Aug 11
Counterfactual Benchmarking and Training for Factuality Consistency and Order-Robust Grounded Reasoning in LLMs over Heterogeneous Knowledge
arXiv:2608. 07838v1 Announce Type: new Abstract: Large language models (LLMs) have increasingly supported response generation grounded in user-provided knowledge spanning heterogeneous structures.
arXiv AI
Jun 19
Detecting Hallucinations for Large Language Model-based Knowledge Graph Reasoning
arXiv:2606. 19351v1 Announce Type: cross Abstract: Knowledge graph (KG) reasoning infers new knowledge from existing facts and is widely applied in question answering, recommendation, and decision support.
arXiv AI
Aug 18
A Large-Scale Chinese Knowledge Graph-Text Alignment Dataset for Benchmarking Knowledge-Grounded LLMs
arXiv:2510. 06039v2 Announce Type: replace-cross Abstract: Reliable evaluation of knowledge-grounded Large Language Models (LLMs) in Chinese requires resources that explicitly align Chinese-language text with verifiable Knowledge Graph (KG) facts.
arXiv AI
Aug 28
Co-Evolving Structured Knowledge and Reasoning in Language Models
The paper introduces KBevo, a co‑evolving framework that simultaneously builds a structured knowledge base and performs reasoning over it for knowledge‑intensive question answering. By optimizing both components end‑to‑end with QA outcome rewards, the system improves the quality and connectivity of the knowledge base, leading to higher answer reachability and better compositional factual reasoning. Compared to standard retrieval baselines, KBevo offers greater controllability and improved factual accuracy.