arXiv Computation and Language
Compound-QA: A Benchmark for Evaluating LLMs on Compound Questions
Read the original on arXiv Computation and Language →The Flow has not summarised this story yet — read it at arXiv Computation and Language.
Related stories
arXiv AI
Jun 26
NuclearQAv2: A Structured Benchmark for Evaluating Domain-Science Competence in Large Language Models
arXiv:2606. 27047v1 Announce Type: cross Abstract: Large language models (LLMs) have demonstrated strong performance across a wide range of tasks, but ensuring their reliability in highly technical domains remains a significant challenge.
arXiv AI
Jul 17
Simplicity Paradox: Debunking myths about prompting and datasets for LLM evaluation
arXiv:2607. 14109v1 Announce Type: cross Abstract: Probing the capabilities of Large Language Models (LLMs) and building robust solutions for Multiple-Choice Question Answering (MCQA) remain central challenges in natural language understanding.
arXiv AI
Jun 24
Benchmarking LLMs' Mathematical Reasoning with Unseen Random Variables Questions
arXiv:2501. 11790v5 Announce Type: replace-cross Abstract: Recent studies have raised significant concerns regarding the reliability of current mathematics benchmarks, highlighting issues such as simplistic design and potential data contamination.
arXiv AI
Aug 11
Beyond Questions: Evaluating LLM's Knowledge Expression
arXiv:2605. 26937v2 Announce Type: replace-cross Abstract: Parametric knowledge in large language models (LLMs) is a cornerstone of their success, yet remains poorly understood.
arXiv AI
Jun 26
Ask, Don't Judge: Binary Questions for Interpretable LLM Evaluation and Self-Improvement
arXiv:2606. 27226v1 Announce Type: new Abstract: Evaluating LLM outputs remains a major bottleneck in NLP: human evaluation is expensive and slow, lexical metrics correlate poorly with human judgments on open-ended generation, and holistic LLM judges often produce opaque scores that are hard to debug.
Hugging Face Trending Papers
Jun 25
Ask, Don't Judge: Binary Questions for Interpretable LLM Evaluation and Self-Improvement
Evaluating LLM outputs remains a major bottleneck in NLP: human evaluation is expensive and slow, lexical metrics correlate poorly with human judgments on open-ended generation, and holistic LLM judges often produce opaque scores that are hard to debug. We propose BINEVAL, a framework that decomposes evaluation criteria into atomic binary questions and aggregates the resulting verdicts into interpretable, multi-dimensional scores.