arXiv AI

CSTutorBench: Benchmarking Small Language Models as Tutors for Block-Based Programming

arXiv:2607. 05571v1 Announce Type: new Abstract: Large language models are increasingly explored as AI tutors, yet deploying them in K-12 settings raises concerns around privacy, cost, and reliance on proprietary models.

arXiv AI
Aug 25

Evaluating Large Language Models for automatic analysis of teacher simulations

The paper evaluates Large Language Models for automatically analyzing responses in digital teacher simulations. Experiments compare DeBERTaV3 and Llama 3 across zero‑shot, few‑shot, and fine‑tuning settings, revealing that performance varies by characteristic and that Llama 3 consistently outperforms DeBERTaV3, especially when new characteristics must be identified. The findings suggest Llama 3 is preferable for dynamic simulation environments where teacher educators introduce new evaluation criteria.

By David de-Fitero-Dominguez, Mariano Albaladejo-Gonz\'alez, Antonio Garcia-Cabot, Eva Garcia-Lopez, Antonio Moreno-Cediel, Erin Barno, Justin Reich
arXiv Computation and Language
Aug 27

EduDial: Constructing a Large-scale Multi-turn Teacher-Student Dialogue Corpus

EduDial is a large-scale multi-turn teacher‑student dialogue corpus covering 345 core knowledge points and 34,250 dialogue sessions, designed around Bloom’s taxonomy and ten questioning strategies such as situational, ZPD, and metacognitive questioning. The dataset includes differentiated teaching strategies for students at varying cognitive levels to provide targeted guidance. Using EduDial, the authors trained EduDial‑LLM 32B and introduced an 11‑dimensional evaluation framework that measures teaching quality and content quality, showing that most mainstream LLMs struggle with student‑centered teaching while EduDial‑LLM outperforms all baselines across all metrics.

By Shouang Wei, Min Zhang, Xin Lin, Bo Jiang, Zhongxiang Dai, Kun Kuang
arXiv AI
Sep 25

Guardrails or Roadblocks? Effects of Pedagogical Style and Context Awareness in AI Teaching Assistants for Programming

The study examined how different designs of AI teaching assistants (AI TAs) affect students in an introductory programming course. Four AI TAs were compared based on pedagogical style (Socratic vs. Direct instruction) and context awareness (no context vs. full context). Results showed that the Socratic AI TA with full context received the lowest favorability ratings, had the highest interaction stress, the most external LLM use, and the lowest comprehension outcomes, though differences were not statistically significant.

By Madeleine Eastwood, Harshith Narne, Joseph Hilby, Paul Denny, Ashish Aggarwal, Amanpreet Kapoor
arXiv AI
Jul 28

A didactical-driven teacher assistant for a dimensional modeling course

arXiv:2607. 22598v1 Announce Type: cross Abstract: Educational chatbots powered by large language models (LLMs) show promising effects on learning outcomes, yet most systems delegate pedagogical decisions such as content selection and didactic structuring implicitly to the LLM, making tutoring strategies difficult to trace, evaluate, and reproduce.

By Laurent Brisson (IMT Atlantique - DSD), Maria Segarra (IMT Atlantique - INFO, Lab-STICC\_MOTEL), Gr\'egory Smits (IMT Atlantique - INFO, Lab-STICC\_MOTEL)
arXiv AI
Sep 15

LoRA Fine-Tuned Models for Control Systems Course Q\&A: A Multidimensional Evaluation of Model Scale and Rank Effects

The paper presents a study on fine‑tuning open‑source instruction‑tuned language models for a Linear Control Systems course using LoRA. A dataset of 360 system‑user‑assistant conversations was created, and LoRA was applied to Qwen2.5‑3B‑Instruct and Qwen2.5‑7B‑Instruct with ranks r=4, 8, and 16. Evaluation with ROUGE, BERTScore, and structured‑output metrics showed that LoRA improved similarity and stability, with the 7B‑r16 model achieving the highest scores and r=8 offering a good trade‑off between performance and parameter efficiency.

By Shaowen Lu, Chengxu Liu, Ping Zhou, Tao Yang
arXiv AI
Jun 9

Lost in the Flow with Code Talkers: Unveiling the Instruction-Tuning Tax of Large Language Models in Code Tasks

arXiv:2606. 08676v1 Announce Type: cross Abstract: AI coding assistants have significantly improved developer productivity by automatically suggesting code that aligns with user intent, and many of these tools are now integrated directly into Integrated Development Environments (IDEs).

By Shi Ying Chang, Chiok Yew Ho, Yichen Li, Yintong Huo
arXiv AI
Aug 5

EduClaw-Bench: A Long-Horizon Benchmark for Pedagogical LLM Agents with Simulated Learners

arXiv:2608. 03206v1 Announce Type: cross Abstract: Large language models (LLMs) power educational applications from tutoring to essay scoring, but each is a point solution to a single task, and only recently have these point solutions been integrated into agents operating over a learning management system (LMS).

By Unggi Lee, Sookbun Lee, Yeil Jeong, Eunjoo Lee, Minchul Shin, Hoilym Kwon