arXiv AI

From Mastery Profile to Simulated Response: Stochastic Student Knowledge Graphs (SSKG) for Faithful LLM Student Simulation

arXiv AI
Aug 5

EduClaw-Bench: A Long-Horizon Benchmark for Pedagogical LLM Agents with Simulated Learners

arXiv:2608. 03206v1 Announce Type: cross Abstract: Large language models (LLMs) power educational applications from tutoring to essay scoring, but each is a point solution to a single task, and only recently have these point solutions been integrated into agents operating over a learning management system (LMS).

By Unggi Lee, Sookbun Lee, Yeil Jeong, Eunjoo Lee, Minchul Shin, Hoilym Kwon
arXiv AI
Sep 10

Tracing Mathematical Proficiency Through Problem-Solving Processes

The paper introduces Knowledge Tracing Leveraging Problem‑Solving Process (KT‑PSP), a method that incorporates students’ problem‑solving steps to model mathematical proficiency more comprehensively than traditional knowledge tracing. It presents the KT‑PSP‑25 dataset and a new framework, StatusKT, which uses a teacher‑student‑teacher LLM pipeline to extract proficiency indicators, generate responses, and evaluate mastery. Experiments show that StatusKT improves prediction accuracy and offers interpretable explanations by explicitly modeling proficiency.

By Jungyang Park, Suho Kang, Jaewoo Park, Jaehong Kim, Jaewoo Shin, Seonjoon Park, Youngjae Yu
arXiv AI
Oct 9

Universal Textual Teaching for LLMs

The paper introduces Universal Textual Teaching (UTT), a parameter‑update‑free framework that distills knowledge gaps between a Teacher and Student large language model into a reusable natural‑language artifact called a Primer. UTT identifies representative gap cases, then iteratively refines the Primer through a cycle of student attempts, prompter‑generated teaching instructions, teacher demonstrations, and synthesizer consolidation. Experiments on math (Omni‑MATH‑2) and code generation (KernelBench) show significant accuracy gains, outperforming prompt engineering and parameter‑based knowledge distillation, and the Primer generalizes across different Teacher‑Student pairs.

By Zhanyi Lu, Huan Wang
arXiv Computation and Language
Sep 2

A Dataset for Modeling Iterative Problem-Solving

The paper introduces CodeInsight, a large-scale dataset of over 3 million code submissions from 3,286 undergraduate students in two introductory C++ courses, capturing test‑case outcomes, timestamps, and source code. It presents a benchmark that evaluates various modeling approaches—including a Recurrent State Space Model and an LLM‑based predictor—on their ability to predict iterative problem‑solving dynamics such as performance changes and error persistence. The study finds that the RSSM outperforms other models on most courses, while the LLM generates full submissions but with lower predictive accuracy, suggesting it functions more as a generative solver than a behavior predictor.

By Fagun Patel, Sang T. Truong, Duc Q. Nguyen, Kazunori Fukuhara, Benjamin W. Domingue, Sanmi Koyejo, Nick Haber
arXiv AI
Aug 25

GIM: Evaluating models via tasks that integrate multiple cognitive domains

The paper introduces the Grounded Integration Measure (GIM), a benchmark of 820 expert‑authored problems designed to test models on tasks that integrate multiple cognitive operations such as constraint satisfaction, state tracking, epistemic vigilance, and audience calibration. GIM emphasizes realistic, broadly accessible knowledge rather than specialized expertise, and uses a judge‑aware 2‑parameter logistic IRT model to produce robust ability estimates across 53 model‑thinking‑level configurations. The authors provide a comprehensive leaderboard of 22 models and 47 test configurations, and conduct an extensive study on how test‑time compute affects model capability, finding that configuration choices like thinking budget and quantization can be as influential as model selection itself. whyItMatters":"By focusing on integration of multiple cognitive domains, GIM offers a more realistic assessment of model reasoning capabilities than benchmarks that either overemphasize memorization or abstract reasoning alone."

By Rohit Patel, Alexandre Rezende, Steven McClain
arXiv Computation and Language
Aug 31

SMRC: Aligning Large Language Models with Student Reasoning for Mathematical Error Correction

SMRC is a new method that aligns large language models with student reasoning for mathematical error correction. It treats student reasoning as a multi‑step decision problem and uses Monte Carlo Tree Search to find optimal correction paths, while a breadth‑first search guided by the model generates reward signals that are back‑propagated to supervise intermediate steps. The authors also introduce the MSEB benchmark of 158 high‑school math problems and a dual evaluation protocol focusing on solution accuracy and correct‑step retention, showing that SMRC outperforms existing methods on several datasets.

By Biaojie Zeng, Min Zhang, Juan Zhou, Fengrui Liu, Ruiyang Huang, Yu Song, Xin Lin