SkillEval introduces a new evaluation framework for large language models that moves beyond single scalar scores to perform cross-model, skill-level analysis. It automatically discovers interpretable skills from benchmark questions using LLM meta‑cognition and applies cognitive diagnosis models to estimate per‑skill abilities. Experiments on 3,811 LLMs across five benchmarks show that SkillEval yields more calibrated, informative model profiles, enabling tasks such as efficient LLM routing and adaptive testing.
By Berke Arda, Peng Cui, Qiaoyuan Zheng, Rudolf Debelak, Mubashara Akhtar, Mrinmaya Sachan
arXiv:2606. 06804v1 Announce Type: new Abstract: Digital learning environments record learners' responses to individual items, making it possible to study the development of specific skills rather than overall scores.
By Yawen Ma, Sahoko Ishida, Kate Cain, Gabriel Wallin
arXiv:2509.05346v3 Announce Type: replace
Abstract: While large language models (LLMs) are increasingly being adopted to support personalized learning, there remains limited understanding of how thei...
By Bo Yuan, Jiazi Hu
The paper introduces LT‑MKT, a new method for multi‑domain knowledge tracing that incorporates cognitive load and knowledge transfer. It constructs a multi‑domain hierarchical graph using textual information from questions and concepts, then explicitly models cross‑domain temporal and knowledge features to capture cognitive load effects. A knowledge transfer module further captures propagation of knowledge states within and across domains, leading to more accurate predictions of students’ future performance.
By Haotian Zhang, Shucun Wang, Jinze Wu, Liang Ding, Shuochen Liu, Zhenya Huang, Jing Sha, Shijin Wang, Qi Liu
The paper explores using semantic text embeddings to automatically build educational taxonomies by clustering math question embeddings. It compares these embedding‑based taxonomies to existing Common Core State Standards (CCSS) taxonomies in two ASSISTments datasets, finding substantial but incomplete overlap. When applied to Deep Knowledge Tracing, the new taxonomies improve predictive performance by about 3‑5 AUC percentage points.
By Kia Karbasi, Kaveh Karbasi, Gregory Pottie
arXiv:2606. 28186v1 Announce Type: cross Abstract: Predicting human item difficulty is central to educational assessment, where reliable estimates support fairness and effective test construction.
By Chenguang Wang, Ming Li, Xinyue Zeng, Zhuochun Li, Hong Jiao, Tianyi Zhou, Dawei Zhou