arXiv AI
Beyond Benchmarking: Scenario-Based Evaluation of Large Language Models for Personalized Learning
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Sep 4
A Prompt-Engineering Approach to Develop Scalable, Flexible, and Real-Time Hybrid Micro-Level Personalization in a General Purpose AI Teaching Assistant
The paper introduces a prompt‑engineering framework that personalizes large language model (LLM) teaching assistants across disciplines by tailoring responses to six learner‑specific dimensions, creating 96 distinct learner profiles. It also analyzes student queries through Bloom’s Taxonomy to gauge cognitive complexity, encoding both learner attributes and cognitive assessments into structured prompts that condition the LLM without retraining. Experiments using NLP metrics and a small human study demonstrate that this approach yields perceptible differences in response style and structure, with statistical evidence linking specific learner attributes to measurable changes.
arXiv Computation and Language
Aug 27
EduDial: Constructing a Large-scale Multi-turn Teacher-Student Dialogue Corpus
EduDial is a large-scale multi-turn teacher‑student dialogue corpus covering 345 core knowledge points and 34,250 dialogue sessions, designed around Bloom’s taxonomy and ten questioning strategies such as situational, ZPD, and metacognitive questioning. The dataset includes differentiated teaching strategies for students at varying cognitive levels to provide targeted guidance. Using EduDial, the authors trained EduDial‑LLM 32B and introduced an 11‑dimensional evaluation framework that measures teaching quality and content quality, showing that most mainstream LLMs struggle with student‑centered teaching while EduDial‑LLM outperforms all baselines across all metrics.
arXiv AI
Aug 7
Evaluating and Improving Pedagogical Fit in LLM-Based AI Tutors with the Pedagogical Suitability Index
arXiv:2608. 05411v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used as AI tutors, but a correct answer is not always a pedagogically appropriate one.
arXiv Machine Learning
Jun 8
Elmes*: Automated Construction of Fine-Grained Evaluation Rubrics for Large Language Models in Long-Tail Educational Scenarios
arXiv:2606. 06546v1 Announce Type: new Abstract: Evaluating large language models (LLMs) for education requires measuring how models teach, not only what they know.
arXiv AI
3d ago
RegulaTutor: A Theory-Informed Framework for Long-Term Personalized Learning
RegulaTutor is a theory-informed framework for long‑term personalized tutoring that links a learner’s historical evidence to instructional support. Guided by self‑regulated learning research, it uses a cognitive, affective, metacognitive, and motivational (CAMM) framework to derive evidence‑supported learner interpretations and teaching policies that are updated as new observations arrive. The framework includes PairRoute, an offline refinement procedure that distinguishes harness, model, and mixed failures, proposes targeted harness revisions, and constructs stage‑level supervision for confirmed model failures, improving historical‑evidence accuracy, learner diagnosis, and teaching quality across evaluated model configurations.
arXiv AI
Jul 10
DeepTutor: Towards Agentic Personalized Tutoring
arXiv:2604. 26962v3 Announce Type: replace-cross Abstract: Education is one of the most promising real-world applications for Large Language Models (LLMs).