← Back to all news
arXiv AI By Zongrui Yang, Haoyuan Li, Zhongsheng Wang, Zhirui Zeng, Pengqian Han, Yi Zhou, Yuting Wang, Jiamou Liu

Chinese Competitive Debating Dataset and Benchmark

Read the original on arXiv AI →

The paper introduces a Chinese competitive debating dataset and benchmark, comprising 148 fully adjudicated matches with 2,698 stages and 20,542 exchange units, all annotated by 120 professional judges using a shared rubric. It offers manually verified transcripts, segmentation, and preserved scores, votes, and rationales, enabling evaluation of large language models on winner-tendency prediction, stage-score prediction, and best-debater prediction. Zero‑shot experiments show the best models achieve 66.2% winner‑prediction accuracy, a 0.250 Pearson correlation for stage scores, and 56.8% best‑debater accuracy.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv Computation and Language
Aug 31

Evaluating the Performance of Large Language Models on GAOKAO Benchmark

arXiv:2305.12474v4 Announce Type: replace Abstract: Large Language Models(LLMs) have demonstrated remarkable performance across various natural language processing tasks; however, how to comprehensiv...

By Xiaotian Zhang, Chunyang Li, Yi Zong, Zhengyu Ying, Liang He, Xipeng Qiu, Tianxiang Sun, Peng Li, Shiqiao Meng, Yanjun Zheng, Jun Zhan, Zhangyue Yin, Xiannian Hu, Guofeng Quan, Qixiang Wang
llmsnlpbenchmarks
More like this →
arXiv Computation and Language
Aug 28

Evaluating Language Models in Realistic Conversational Contexts

The paper introduces UPHELD, a large benchmark of human-to-human dialogues written by professional script writers, featuring realistic turn densities and over 36,000 per-turn human annotations. It evaluates existing automatic metrics and LLM-as-a-judge methods, finding them unreliable against expert human judgment. Using UPHELD, the authors develop a Mixture-of-Judges framework that improves correlation with human assessments by about 30%.

By Ilija Subasic, Andrew Rabinovich, Zhao Chen
llmsnlpbenchmarks
More like this →
arXiv Computation and Language
Sep 16

Toward Robust LLM-Based Judges: Taxonomic Bias Evaluation and Debiasing Optimization

The paper introduces JudgeBiasBench, a benchmark that systematically quantifies judgment biases in large language model (LLM)-based judges across four dimensions and 12 bias types. It evaluates both generative and discriminative judges, revealing significant bias patterns that undermine reliability. The authors propose bias-aware training—reinforcement learning for generative judges and contrastive learning for discriminative judges—to reduce these biases while maintaining evaluation performance.

By Hongli Zhou, Hui Huang, Rui Zhang, Kehai Chen, Bing Xu, Conghui Zhu, Tiejun Zhao, Muyun Yang
llmsreinforcement-learningbenchmarkssafety
More like this →