Hugging Face Trending Papers June 17, 2026

LLMs Struggle to Measure What Distinguishes Students of Different Proficiency Levels: A Study of Item Discrimination in Reading Comprehension Assessment

Read the original on Hugging Face Trending Papers →

Item discrimination is a fundamental psychometric property of educational assessment, which measures whether an item meaningfully distinguishes students with higher proficiency from students with lower proficiency. While various existing works have explored whether large language models (LLMs) can estimate item difficulty, it remains unclear whether they can capture item discrimination.

Summary generated by The Flow from the publisher's feed. The full article lives at Hugging Face Trending Papers.

arXiv AI

Jul 28

Reconstructing Item Characteristic Curves using Fine-Tuned Large Language Models

arXiv:2601. 02580v2 Announce Type: replace-cross Abstract: Traditional methods for determining assessment item parameters, such as difficulty and discrimination, rely heavily on expensive field testing to collect student performance data for Item Response Theory (IRT) calibration.

By Christopher Ormerod

llms fine-tuning

arXiv AI

Jul 15

Adaptive Testing for LLM Evaluation: A Psychometric Alternative to Static Benchmarks

arXiv:2511. 04689v3 Announce Type: replace-cross Abstract: Evaluating large language models (LLMs) typically requires thousands of benchmark items, making the process expensive, slow, and increasingly impractical at scale.

By Peiyu Li, Xiuxiu Tang, Si Chen, Ying Cheng, Ronald Metoyer, Ting Hua, Nitesh V. Chawla

llms benchmarks

arXiv AI

Jul 29

Laplace-PSN-IRT: Uncertainty Quantification for Neural Item Response Theory Models of LLM Benchmarks

arXiv:2607. 25257v1 Announce Type: cross Abstract: Item Response Theory (IRT) has recently been proposed as a framework for evaluating large language model (LLM) benchmarks by separating a model's latent ability from the properties of individual benchmark items.

By Juan Francisco, Mandujano Reyes

llms benchmarks

arXiv Machine Learning

Jun 2

How to Correctly Report LLM-as-a-Judge Evaluations

arXiv:2511. 21140v4 Announce Type: replace Abstract: Large language models (LLMs) are widely used as scalable evaluators of model responses in lieu of human annotators.

By Chungpa Lee, Thomas Zeng, Jongwon Jeong, Jy-yong Sohn, Kangwook Lee

llms safety

arXiv AI

Jul 31

Aligning LLM-Simulated and Human Examinees for Psychometric Calibration: A Cognitive Diagnostic Profiling Approach

arXiv:2607. 26317v1 Announce Type: cross Abstract: Psychometric calibration for educational tests typically requires costly human response data.

By Wenjie Zhou, Yunting Liu, Renjiao Tang, Mark Wilson

llms safety

arXiv AI

Aug 6

Revisiting Generalization Across Difficulty Levels: It's Not So Easy

arXiv:2511. 21692v3 Announce Type: replace-cross Abstract: We investigate how well large language models (LLMs) generalize across different task difficulties, a key question for effective data curation and evaluation.

By Yeganeh Kordi, Nihal V. Nayak, Max Zuo, Ilana Nguyen, Stephen H. Bach

llms

Related stories

Reconstructing Item Characteristic Curves using Fine-Tuned Large Language Models

Adaptive Testing for LLM Evaluation: A Psychometric Alternative to Static Benchmarks

Laplace-PSN-IRT: Uncertainty Quantification for Neural Item Response Theory Models of LLM Benchmarks

How to Correctly Report LLM-as-a-Judge Evaluations

Aligning LLM-Simulated and Human Examinees for Psychometric Calibration: A Cognitive Diagnostic Profiling Approach

Revisiting Generalization Across Difficulty Levels: It's Not So Easy