arXiv Computation and Language
Oct 7

Agreement Is Not Validity: Cross-Model LLM Consensus in Diagnosing Student Failure Modes in K-12 Math Tutoring Dialogue

arXiv:2610.08703v1 Announce Type: new Abstract: In K-12 mathematics tutoring, student-tutor dialogue provides rich evidence of learners' problem-solving processes and sources of difficulty. Learning...

By Clayton Cohn, Joyce Fonteles, Kirk Vanacore, Gianni Mazza, Candida Crawford, Tom Hooper, Gautam Biswas, Rene Kizilcec
arXiv AI
2d ago

TrussTutor-Bench: Benchmarking large language models on solution accuracy, sycophancy, and error detection in statics

TrussTutor-Bench is a benchmark of 200 verified two-dimensional truss problems with automatically generated student solutions, designed to evaluate large language models on solution accuracy, sycophancy, and error detection. Thirteen LLMs were tested, showing accuracy ranging from 64.5% to 97.0%, near-perfect error detection (90.4%–100%), and varying degrees of sycophancy (1.6%–50.0% of correct solutions rejected when a student doubted them). The benchmark highlights wide performance differences among models, especially in handling student doubt, and offers a verified, extensible standard for assessing technical reliability before deploying LLMs in educational settings.

By Evangelos Apostolakis, Erfan Hamdi, Sara Jahedazad, Emma Lejeune
arXiv AI
Oct 9

Grammar Concept Annotation at Scale: Deployed Fine-Tuned Small Language Models Outperform Prompted Frontier Models

The paper reports that fine‑tuned small language models (SLMs) can outperform large frontier models in annotating grammar concepts from learner–tutor transcripts. By training a 0.8B Qwen3.5 model on teacher‑generated data and embedding the annotation rules into adapter weights, the authors deploy an efficient end‑to‑end grammar mastery tracker that matches or exceeds the performance of 4B reference models and GPT‑5.4/5.6 on two human‑curated benchmarks. The lightweight model also cuts serving costs by about 16× and boosts learner engagement and key business metrics in an online experiment.

By Marjan Celikik, Ana Peleteiro Ramallo, Javier Morales