Hugging Face Trending Papers July 8, 2026

From Text to Parameters: Predicting Item Parameters from Embedding Regularization with Reliability and Design Ceilings

Read the original on Hugging Face Trending Papers →

Newly developed items must ordinarily be field tested before their psychometric properties are known, creating a cold start problem for item calibration. Predicting item parameters from features is a long standing measurement problem dating back to the Linear Logistic Test Model; modern text embeddings now automate the design matrices traditionally specified by hand.

Summary generated by The Flow from the publisher's feed. The full article lives at Hugging Face Trending Papers.

arXiv Machine Learning

Aug 3

Can LLMs Really Understand Item Difficulty Levels? Implications for Automated Item Generation Using LLMs

arXiv:2607. 28634v1 Announce Type: cross Abstract: The estimation of item difficulty plays a key role in both formative assessment and large-scale high-stakes summative assessments.

By Xinyi Wang, Hong Jiao, Ming Li, Sydney Peters, Hanna Choi, Tianyi Zhou, Qingshu Xu

llms rag

arXiv AI

Jul 15

Adaptive Testing for LLM Evaluation: A Psychometric Alternative to Static Benchmarks

arXiv:2511. 04689v3 Announce Type: replace-cross Abstract: Evaluating large language models (LLMs) typically requires thousands of benchmark items, making the process expensive, slow, and increasingly impractical at scale.

By Peiyu Li, Xiuxiu Tang, Si Chen, Ying Cheng, Ronald Metoyer, Ting Hua, Nitesh V. Chawla

llms benchmarks

arXiv AI

Jul 28

Reconstructing Item Characteristic Curves using Fine-Tuned Large Language Models

arXiv:2601. 02580v2 Announce Type: replace-cross Abstract: Traditional methods for determining assessment item parameters, such as difficulty and discrimination, rely heavily on expensive field testing to collect student performance data for Item Response Theory (IRT) calibration.

By Christopher Ormerod

llms fine-tuning

arXiv AI

Aug 10

Automated item evaluation: Predicting item acceptance and rejection using LLM-generated critiques

arXiv:2608. 06609v1 Announce Type: new Abstract: Automated item evaluation (AIE) refers to the use of computational methods to assess item quality without requiring manual expert review or field testing of the items under evaluation.

By Hotaka Maeda, Yikai Lu

llms fine-tuning safety

arXiv AI

Aug 6

Item Response Theory for AI Safety

arXiv:2608. 05086v1 Announce Type: new Abstract: Language models differ in how safely they behave and these differences are measured by safety benchmarks.

By Joshua Fonseca Rivera (Independent), Neil Shah (Independent), David Demitri Africa (UK AI Security Institute), Konstantinos Voudouris (UK AI Security Institute)

llms benchmarks safety

arXiv AI

Jul 20

Latency-Response Theory Model: Evaluating Large Language Models via Response Accuracy and Chain-of-Thought Length

arXiv:2512. 07019v3 Announce Type: replace-cross Abstract: The proliferation of Large Language Models (LLMs) necessitates valid evaluation methods to provide guidance for both downstream applications and actionable future improvements.

By Zhiyu Xu, Jia Liu, Yixin Wang, Yuqi Gu

llms benchmarks

Related stories

Can LLMs Really Understand Item Difficulty Levels? Implications for Automated Item Generation Using LLMs

Adaptive Testing for LLM Evaluation: A Psychometric Alternative to Static Benchmarks

Reconstructing Item Characteristic Curves using Fine-Tuned Large Language Models

Automated item evaluation: Predicting item acceptance and rejection using LLM-generated critiques

Item Response Theory for AI Safety

Latency-Response Theory Model: Evaluating Large Language Models via Response Accuracy and Chain-of-Thought Length