arXiv Computation and Language

TutorCaseBench: A Case-Grounded Benchmark for Evaluating LLM Tutors in Supporting Students with Challenging Behaviors

TutorCaseBench is a new benchmark designed to evaluate large language model (LLM) tutors on their ability to support students exhibiting challenging behaviors. It is built from qualitative case studies in pedagogy, educational psychology, and special education, converting documented student categories into 923 case-specific dialogues and rubrics. The benchmark tests 11 state‑of‑the‑art LLMs on open‑ended response generation and multiple‑choice selection, finding that even the best model meets rubric criteria only 55.8% of the time for open‑ended cases, highlighting a gap between recognizing interventions and adapting to context.

arXiv Computation and Language
Aug 25

LLM Pedagogical Behavior in AI Tutoring Interactions

arXiv:2608.22993v1 Announce Type: new Abstract: Students increasingly use LLMs as tutors for coursework and problem solving. Little is known about the level of assistance LLMs provide when students u...

By Suhyeon Lee, Juneha Baek, Jaehyeong Park, Donghyuk Shin
arXiv AI
Aug 19

Effective Personalized AI Tutors via LLM-Guided Reinforcement Learning

The paper presents a tutoring platform that combines a generative AI chatbot with a reinforcement learning algorithm to adaptively sequence practice problems for students learning Python. In a five‑month field study across ten high schools, the adaptive sequencing improved unassisted final exam performance by 0.15 standard deviations, with mediation analysis indicating that higher engagement drove the gains. The study demonstrates that signals from student‑chatbot interactions can be leveraged to personalize and optimize learning at scale.

By Angel Tsai-Hsuan Chung, Botong Zhang, Ling-Chieh Kung, Hamsa Bastani, Osbert Bastani
arXiv AI
Aug 28

TutorTrace: A Dataset and Taxonomy for Classifying Learner Behavioral States during AI-Assisted Programming Education

TutorTrace is a new dataset and behavioral abstraction pipeline that captures learners’ low‑level IDE telemetry to make their behavioral context visible and computable in real time. The dataset, collected across 480 students in two introductory Python courses, includes 180 K telemetry events, 13 633 behavioral segments, and 27 continuously computed metrics, and it underpins a taxonomy of learner activity before, between, and after AI queries. Preliminary classroom tests show that behavior‑aware prompts reduce the time between queries, and the system can predict upcoming queries with AUROC scores of .726 and .717 on two held‑out tasks.

By David Barron, Xiaohang Tang, Rezky Dwisantika, Minsun Kim, David H. Smith IV, Jiaming Cui, Yan Chen
arXiv AI
5d ago

Interpretable Difficulty-Aware Knowledge Tracing in Tutor-Student Dialogues

The paper introduces an interpretable, difficulty-aware knowledge tracing framework for tutor‑student dialogues that leverages large language models. It explicitly models student ability and question difficulty at each turn, using the original question text and the next tutor‑posed task to estimate knowledge state. By integrating Item Response Theory, the framework translates LLM outputs into cognitive‑theory‑based parameters, achieving superior performance over existing baselines while producing interpretable predictions.

By Shuyan Huang, Alexander Scarlatos, Jaewook Lee, Andrew Lan