SkillEval introduces a new evaluation framework for large language models that moves beyond single scalar scores to perform cross-model, skill-level analysis. It automatically discovers interpretable skills from benchmark questions using LLM meta‑cognition and applies cognitive diagnosis models to estimate per‑skill abilities. Experiments on 3,811 LLMs across five benchmarks show that SkillEval yields more calibrated, informative model profiles, enabling tasks such as efficient LLM routing and adaptive testing.
By Berke Arda, Peng Cui, Qiaoyuan Zheng, Rudolf Debelak, Mubashara Akhtar, Mrinmaya Sachan
arXiv:2606. 06804v1 Announce Type: new Abstract: Digital learning environments record learners' responses to individual items, making it possible to study the development of specific skills rather than overall scores.
By Yawen Ma, Sahoko Ishida, Kate Cain, Gabriel Wallin
arXiv:2509.05346v3 Announce Type: replace
Abstract: While large language models (LLMs) are increasingly being adopted to support personalized learning, there remains limited understanding of how thei...
By Bo Yuan, Jiazi Hu
The paper introduces LT‑MKT, a new method for multi‑domain knowledge tracing that incorporates cognitive load and knowledge transfer. It constructs a multi‑domain hierarchical graph using textual information from questions and concepts, then explicitly models cross‑domain temporal and knowledge features to capture cognitive load effects. A knowledge transfer module further captures propagation of knowledge states within and across domains, leading to more accurate predictions of students’ future performance.
By Haotian Zhang, Shucun Wang, Jinze Wu, Liang Ding, Shuochen Liu, Zhenya Huang, Jing Sha, Shijin Wang, Qi Liu
The paper explores using semantic text embeddings to automatically build educational taxonomies by clustering math question embeddings. It compares these embedding‑based taxonomies to existing Common Core State Standards (CCSS) taxonomies in two ASSISTments datasets, finding substantial but incomplete overlap. When applied to Deep Knowledge Tracing, the new taxonomies improve predictive performance by about 3‑5 AUC percentage points.
By Kia Karbasi, Kaveh Karbasi, Gregory Pottie
arXiv:2606. 28186v1 Announce Type: cross Abstract: Predicting human item difficulty is central to educational assessment, where reliable estimates support fairness and effective test construction.
By Chenguang Wang, Ming Li, Xinyue Zeng, Zhuochun Li, Hong Jiao, Tianyi Zhou, Dawei Zhou
The paper introduces TIZSKT, a zero‑shot cross‑domain knowledge tracing model that separates behavioral sequence modeling from domain‑local historical evidence construction. It uses text as an index via a frozen text encoder and unsupervised per‑domain projection to build question addresses, then aggregates a student’s past outcomes into a question‑specific historical correctness summary. Experiments on three public datasets show that TIZSKT outperforms baselines in mean AUC and accuracy, with ablations confirming the importance of text addressing and cross‑segment continuation.
By Yin Yi, Haiping Ma, Siyu Song, Ziwen Wang, Shangshang Yang
Inverse decision modeling infers latent properties of decision processes from observed behavior, but existing formulations rely primarily on action trajectories. In verbalized cognitive tasks, task execution also produces response dynamics that action-only formulations leave unmodeled, such as verbal production, interaction, and hesitation.
arXiv:2608.09222v2 Announce Type: replace
Abstract: Inverse decision modeling infers latent properties of decision processes from observed behavior, but existing formulations rely primarily on action...
By Jiawen Kang, Dongrui Han, Xixin Wu, Helen Meng
CLEAN (Continual Learning with Expandable and Architecturally Isolated Networks) is an incremental cognitive diagnosis framework that allows the concept space to grow while guaranteeing that historical diagnoses remain unchanged. It achieves this by freezing past diagnostic functions, applying orthogonal column masking to prevent gradient interference, and adding expandable full‑rank branches with micro‑variance initialization for new concepts. Experiments on three large‑scale educational datasets show that CLEAN attains zero Representation Drift, preserving old‑item metrics exactly while performing competitively on new items.
By Tao He, Jinxing Xiang, Fan Jiang
The paper investigates when in‑context learning (ICL) can succeed even if the instruction or target task is not covered by a model’s pretraining data. It introduces a theoretical framework that distinguishes two types of distribution shift—unsupported instructions and unsupported target tasks—and shows how demonstrations can mitigate these shifts. The authors further extend the analysis to long‑horizon tasks that can be decomposed into pretraining‑supported subtasks, providing error bounds that explain how ICL can handle tasks beyond the model’s original support.
By Xuhan Tong, Yuchen Zeng, Jiawei Zhang
arXiv:2606. 29049v1 Announce Type: new Abstract: Knowledge Tracing (KT) is important for personalized education but traditionally suffers from two key limitations: a reliance on shallow ID-based representations that neglect semantic depth and a restriction to single-granularity mastery estimation that overlooks hierarchical knowledge dependencies.
By Xinjin Li, Mengyue Wang, Yuzhen Lin, Pengbin Feng, Ziqi Sha, Yeyang Zhou, Yu Ma