← Back to all news
arXiv Computation and Language By Jiashu Yang, Ningning Wang, Yian Zhao, Chaoran Feng, Junjia Du, Hao Pang, Miao Yu, Xuxin Cheng

Kongzi: A Historical Large Language Model with Fact Enhancement

Read the original on arXiv Computation and Language →

Kongzi is a historical large language model that enhances factual accuracy through continued pre‑training on historical corpora, supervised fine‑tuning, and fact‑enhanced reinforcement learning. Its reward system compares entities and relations in generated responses with historical references via knowledge‑graph similarity. Evaluations on 0.5B and 7B models show that Kongzi‑7B achieves the highest historical‑accuracy ratings among compared models and that reinforcement learning improves answer scores and CMMLU‑Chinese‑History accuracy, though it still falls short of the strongest baselines.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

Related stories

arXiv AI
Sep 10

Evaluating and Improving Evidence-Grounded Fact-Checking in LLMs via Multi-Round Evidence Ablation

The paper introduces Fact-Ablated Evaluation (FAE), a framework that iteratively removes cited evidence to test whether large language models (LLMs) adjust their fact‑checking predictions accordingly. Experiments reveal that many off‑the‑shelf LLMs rely more on internal knowledge than on the provided evidence. To address this, the authors propose REAL, a training method that uses counterfactual evidence supervision to encourage LLMs to base veracity judgments on evidence, achieving better evidence‑dependent performance across four datasets.

By Xingyu Deng, Mingzi Cao, Nikolaos Aletras, Xi Wang, Mark Stevenson
llmsfine-tuningbenchmarks
More like this →
arXiv AI
Sep 10

Evaluating the Scalability and Adversarial Generalization of GRPO-Trained NLI Models

The paper evaluates the scalability and adversarial generalization of Natural Language Inference (NLI) models trained with Group Relative Policy Optimization (GRPO) for Chain-of-Thought learning. By fine‑tuning 7B, 14B, and 32B language models with LoRA and QLoRA, the authors show strong performance on standard and adversarial NLI benchmarks, with the 32B model outperforming supervised baselines on adversarial sets. Using AWQ quantization, the 32B model fits within 22 GB of CUDA memory, demonstrating a scalable, practical framework for robust NLI without sacrificing inference quality.

By Pablo Miralles-Gonz\'alez, Javier Huertas-Tato, Alejandro Mart\'in, David Camacho
llmsnlpreinforcement-learningfine-tuningefficiencybenchmarkssafety
More like this →