arXiv AI

Evaluating and Improving Pedagogical Fit in LLM-Based AI Tutors with the Pedagogical Suitability Index

arXiv:2608. 05411v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used as AI tutors, but a correct answer is not always a pedagogically appropriate one.

arXiv Computation and Language
Aug 25

LLM Pedagogical Behavior in AI Tutoring Interactions

arXiv:2608.22993v1 Announce Type: new Abstract: Students increasingly use LLMs as tutors for coursework and problem solving. Little is known about the level of assistance LLMs provide when students u...

By Suhyeon Lee, Juneha Baek, Jaehyeong Park, Donghyuk Shin
arXiv Machine Learning
Sep 14

Simulating Disengaged Students to Evaluate LLM-based Tutors

The paper introduces Disengagement-Aware Student Simulators (DAS2), a protocol that models five learner-engagement states—engaged, gaming, wheel-spinning, off-task, and mixed—to evaluate AI tutor performance before deployment. Using annotated tutoring sessions from ASSISTments09, DAS2’s rule-based labels matched human consensus in 81% of cases, and conditioning simulations on intended states narrowed the correctness-rate gap between simulated and authentic sessions for gaming and wheel-spinning behaviors. The study also compares five AI tutors across these states, finding stable relative rankings but state-specific performance differences, and notes that automated evaluation does not fully align with human judgment.

By Xianghui Meng, Jionghao Lin
arXiv AI
Aug 25

AI University: An LLM-Powered Learning Assistant for Engineering---A Finite Element Method Case Study

AI University (AI‑U) is a flexible framework that uses a fine‑tuned large language model (LLM) combined with retrieval‑augmented generation (RAG) and a reasoning synthesis model to produce style‑aligned responses from lecture videos, notes, and textbooks. In a graduate‑level finite‑element‑method (FEM) course, the authors created a pipeline to generate course‑grounded training data, fine‑tuned an open‑source LLM with Low‑Rank Adaptation (LoRA), and applied RAG‑based synthesis. Evaluation through cosine similarity, LLM‑based assessment, expert review, and user studies showed that the expert model outperformed the base model in alignment with course materials, with 86 % of test cases scoring higher and human users preferring the expert model roughly twice as often. whyItMatters":"The study demonstrates a practical method for building course‑specific learning assistants that improve alignment with instructional content, offering a template that can be extended across STEM fields."

By Mostafa Faghih Shojaei, Rahul Gulati, Benjamin A. Jasperson, Shangshang Wang, Simone Cimolato, Manas Vardhan, Dangli Cao, Willie Neiswanger, Krishna Garikipati
arXiv AI
3d ago

Examining Variation in How Guided AI Tutors Resolve Student Impasses

The study analyzes 20,462 student turns from 1,260 sessions with a guided LLM chemistry tutor, identifying 6,630 impasse turns categorized as conceptual errors, expressed uncertainty, or help‑seeking. Three tutoring conditions—baseline, no‑direct‑answer, and guided—were simulated, revealing that the baseline tutor often gave direct answers, the no‑direct‑answer tutor always asked follow‑up questions, and the guided tutor varied its responses based on context. Impasse trajectories showed that each additional impasse turn reduced the likelihood of recovery, while addressing errors became increasingly beneficial compared to repeated scripted questioning.

By Bakhtawar Ahtisham, Kirk Vanacore, Alessandra Napoli, Josh Arens, Ksenia Ionova, Clayton Cohn, Shima Salehi, Rene Kizilcec
arXiv AI
Jul 16

Learning Engagement Assistant (LEA): Cross-Course Scalability and Classroom Evaluation of an Agentic AI Tutoring System

arXiv:2607. 13370v1 Announce Type: cross Abstract: This paper is an extension of a paper presented at the ICAART 2026 conference, which introduced LEA (Learning Engagement Assistant), an adaptive AI tutoring agent combining course-specific Retrieval-Augmented Generation (RAG) with structured Knowledge Component (KC) models across integrated Chat, Tutor, and Quiz modes.

By Teri Rumble, Javad Zarrin, P. George Lovell, Ruth Falconer