SDUs DAISY is a factual knowledge benchmark focused on Danish cultural heritage, drawing topics from the Danish Culture Canon 2006. The dataset contains 741 manually verified closed‑ended question‑answer pairs generated by querying Wikipedia pages for each canon artifact, covering a wide historical span from 1300 BCE to contemporary pop music, design, and architecture. Baseline tests with state‑of‑the‑art language models show very low performance (best BLEU 0.17, F1 0.27), highlighting the benchmark’s difficulty and the need for improved cultural knowledge in AI systems.
Quiz rooms, trivia nights, and quiz shows challenge human knowledge across a wide range of topics, from canonical facts to everyday culture. In this paper, we examine whether large language models (LLMs) can perform competitively in such settings, using quiz-style questions to test them on both common and niche topics.
The paper compares Knowledge-Graph Based Augmentation (Graph-RAG) with Retrieval-Augmented Generation (RAG) for answering culturally specific questions. Using the LatamQA dataset, Graph-RAG, built automatically from Wikipedia via KGGen, matches RAG performance and reduces the base LLM’s error by 72% with a standard KG and 78% with a benchmark-aware variant. The approach also transfers zero‑shot to Portuguese, showing multilingual applicability.
The paper evaluates the tablet‑2 long‑term memory engine on multilingual text benchmarks and cross‑lingual photo retrieval without lexical matching. Tablet‑2 achieves high accuracy on LongMemEval‑S (95.7%) and moderate accuracy on BEAM‑1M (67.5%), with minimal variance across runs. In multimodal tests, it outperforms BM25 on image‑cell recall and shows significant language‑dependent performance gaps, especially for low‑resource languages.
arXiv:2510.18368v2 Announce Type: replace Abstract: We present $\textbf{Korean SimpleQA (KoSimpleQA)}$, a benchmark for evaluating factuality in large language models (LLMs) with a focus on Korean cu...
We measure tablet-2, a production long-term memory engine for language models, on the text benchmarks the field already uses and on cross-lingual retrieval of photographs stored with no text at all. I...
arXiv:2606. 19727v1 Announce Type: cross Abstract: Language models have become essential tools in shaping modern workflows.
arXiv:2608. 02486v1 Announce Type: cross Abstract: Open-source LLMs reliably name Zeus, Jupiter, and Thor, but recover their counterparts in less-represented traditions like Finnish, Slavic, Egyptian, or Chinese mythology far less consistently.
arXiv:2608. 11741v1 Announce Type: cross Abstract: The scholarly exegesis of ancient Chinese characters demands integrating visual observation, linguistic analysis, and historical context.
arXiv:2601. 14063v2 Announce Type: replace-cross Abstract: Cross-cultural competence in large language models (LLMs) requires understanding and adapting Culture-Specific Items (CSIs) across varying cultural contexts.
The scholarly exegesis of ancient Chinese characters demands integrating visual observation, linguistic analysis, and historical context. However, existing computational approaches focus narrowly on subtasks such as character recognition and retrieval, lacking the structured datasets and benchmarks required for comprehensive scholarly analysis.
Ancient-Bench is a new benchmark for recognizing text on ancient Chinese artifacts, comprising 2,700 images that span 3,000 years of character evolution, nine artifact categories, and seven historical script forms. It introduces three annotation standards—symbol, character, and parsing standardization—to accommodate medium‑specific characteristics and enable consistent evaluation. Experiments show that current Vision‑Language Models and OCR specialists still struggle with variant characters, specialized symbols, and hallucination, indicating the task remains largely unsolved.
arXiv:2608. 15535v1 Announce Type: cross Abstract: We present L3Cube-IndicQuest v2, a large-scale gold-standard multilingual question-answering benchmark for evaluating the India-specific factual knowledge of Large Language Models (LLMs).