arXiv Computation and Language
Evaluating the Performance of Large Language Models on GAOKAO Benchmark
Read the original on arXiv Computation and Language →The Flow has not summarised this story yet — read it at arXiv Computation and Language.
Related stories
arXiv AI
Jun 10
RankLLM: Weighted Ranking of LLMs by Quantifying Question Difficulty
arXiv:2602. 12424v2 Announce Type: replace-cross Abstract: Benchmarks establish a standardized evaluation framework to systematically assess the performance of large language models (LLMs), facilitating objective comparisons and driving advancements in the field.
arXiv AI
Oct 15
Polyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation
arXiv:2409.00696v4 Announce Type: replace-cross Abstract: Rating-based human evaluation has become an essential tool to accurately evaluate the impressive performance of large language models (LLMs)....
arXiv AI
Sep 21
Chinese Competitive Debating Dataset and Benchmark
The paper introduces a Chinese competitive debating dataset and benchmark, comprising 148 fully adjudicated matches with 2,698 stages and 20,542 exchange units, all annotated by 120 professional judges using a shared rubric. It offers manually verified transcripts, segmentation, and preserved scores, votes, and rationales, enabling evaluation of large language models on winner-tendency prediction, stage-score prediction, and best-debater prediction. Zero‑shot experiments show the best models achieve 66.2% winner‑prediction accuracy, a 0.250 Pearson correlation for stage scores, and 56.8% best‑debater accuracy.
Hugging Face Trending Papers
Sep 16
TeochewBench: A Human-Reviewed Benchmark for Teochew Hanzi Translation
Teochew has a substantial speaker community and exhibits distinctive lexical, syntactic, and pragmatic features, yet textual resources for evaluating large language models remain limited. We present T...
arXiv Computation and Language
Sep 16
EviSI: An Evidence-Based Evaluation Agent for Simultaneous Interpreting
EviSI is an evidence‑based evaluation agent for low‑latency simultaneous speech‑to‑speech translation. It combines Multidimensional Quality Metrics with interpreter‑developed criteria, using shared source evidence to assess four dimensions—Anchor, Event, Logic, and Fluency—while deduplicating verified errors before scoring. On English‑to‑Chinese and Chinese‑to‑English data, EviSI’s rankings correlate strongly with human judgments, outperforming BLEU and COMET, and its multilingual extension maintains these correlations across five language directions.
arXiv Computation and Language
Sep 15
When Consistency Does Not Mean Reliability: Evaluating Local LLM Judges Against Human Ratings
arXiv:2609.13824v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used to evaluate the responses of other language models. This approach, known as LLM-as-a-Judge, is faste...