← Back to all news
arXiv Machine Learning By Xianghui Meng, Jionghao Lin

Simulating Disengaged Students to Evaluate LLM-based Tutors

Read the original on arXiv Machine Learning →

The paper introduces Disengagement-Aware Student Simulators (DAS2), a protocol that models five learner-engagement states—engaged, gaming, wheel-spinning, off-task, and mixed—to evaluate AI tutor performance before deployment. Using annotated tutoring sessions from ASSISTments09, DAS2’s rule-based labels matched human consensus in 81% of cases, and conditioning simulations on intended states narrowed the correctness-rate gap between simulated and authentic sessions for gaming and wheel-spinning behaviors. The study also compares five AI tutors across these states, finding stable relative rankings but state-specific performance differences, and notes that automated evaluation does not fully align with human judgment.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Computation and Language
Aug 25

LLM Pedagogical Behavior in AI Tutoring Interactions

arXiv:2608.22993v1 Announce Type: new Abstract: Students increasingly use LLMs as tutors for coursework and problem solving. Little is known about the level of assistance LLMs provide when students u...

By Suhyeon Lee, Juneha Baek, Jaehyeong Park, Donghyuk Shin
llms
More like this →
arXiv AI
Jul 16

Learning Engagement Assistant (LEA): Cross-Course Scalability and Classroom Evaluation of an Agentic AI Tutoring System

arXiv:2607. 13370v1 Announce Type: cross Abstract: This paper is an extension of a paper presented at the ICAART 2026 conference, which introduced LEA (Learning Engagement Assistant), an adaptive AI tutoring agent combining course-specific Retrieval-Augmented Generation (RAG) with structured Knowledge Component (KC) models across integrated Chat, Tutor, and Quiz modes.

By Teri Rumble, Javad Zarrin, P. George Lovell, Ruth Falconer
ragagents
More like this →
arXiv AI
Aug 28

TutorTrace: A Dataset and Taxonomy for Classifying Learner Behavioral States during AI-Assisted Programming Education

TutorTrace is a new dataset and behavioral abstraction pipeline that captures learners’ low‑level IDE telemetry to make their behavioral context visible and computable in real time. The dataset, collected across 480 students in two introductory Python courses, includes 180 K telemetry events, 13 633 behavioral segments, and 27 continuously computed metrics, and it underpins a taxonomy of learner activity before, between, and after AI queries. Preliminary classroom tests show that behavior‑aware prompts reduce the time between queries, and the system can predict upcoming queries with AUROC scores of .726 and .717 on two held‑out tasks.

By David Barron, Xiaohang Tang, Rezky Dwisantika, Minsun Kim, David H. Smith IV, Jiaming Cui, Yan Chen
More like this →
Hugging Face Trending Papers
Jul 15

Learning Engagement Assistant (LEA): Cross-Course Scalability and Classroom Evaluation of an Agentic AI Tutoring System

This paper is an extension of a paper presented at the ICAART 2026 conference, which introduced LEA (Learning Engagement Assistant), an adaptive AI tutoring agent combining course-specific Retrieval-Augmented Generation (RAG) with structured Knowledge Component (KC) models across integrated Chat, Tutor, and Quiz modes. That prior work validated LEA on a single STEM course (CMP511) exclusively through simulation, using synthetic learner agents.

ragagents
More like this →