arXiv AI

When Youth Enter The Chat: An Epistemic Shift in the Validation of LLM-Based Measures of Student Talk

The article discusses the growing use of large language models (LLMs) to assess student discourse at scale, noting that current validation methods—such as expert annotations and F1 scores—often ignore the contextual and cultural nuances of student language. It argues that these practices inadequately capture the experiences of racially and linguistically marginalized youth, and proposes re‑contextualizing classroom conversations and involving students as epistemic authorities. A case study with multilingual 8th‑grade math students demonstrates misalignments between student self‑interpretations and LLM outputs, underscoring the need for youth participation in validating LLM‑based measures.

arXiv Computation and Language
Aug 27

EduDial: Constructing a Large-scale Multi-turn Teacher-Student Dialogue Corpus

EduDial is a large-scale multi-turn teacher‑student dialogue corpus covering 345 core knowledge points and 34,250 dialogue sessions, designed around Bloom’s taxonomy and ten questioning strategies such as situational, ZPD, and metacognitive questioning. The dataset includes differentiated teaching strategies for students at varying cognitive levels to provide targeted guidance. Using EduDial, the authors trained EduDial‑LLM 32B and introduced an 11‑dimensional evaluation framework that measures teaching quality and content quality, showing that most mainstream LLMs struggle with student‑centered teaching while EduDial‑LLM outperforms all baselines across all metrics.

By Shouang Wei, Min Zhang, Xin Lin, Bo Jiang, Zhongxiang Dai, Kun Kuang
arXiv AI
Sep 7

Who Should Grade My Work? Student Perspectives on Transparent AI-Assisted Writing Assessment in Higher Education

The study explores how undergraduate computing students in Saudi Arabia perceive AI‑generated writing feedback when they are explicitly told that ChatGPT, not a human instructor, produced the score and comments. Through qualitative reflections, four themes emerged: students found the feedback useful for surface‑level revisions, recognized AI’s contextual and pedagogical limits, trusted the feedback conditionally—separating its utility from its authority—and reaffirmed the human instructor’s role as the ultimate grading authority. The findings highlight a clear distinction students make between feedback usefulness and evaluative authority, treating them as separate judgments rather than opposing ends of a single approval scale.

By Rayed AlGhamdi
arXiv AI
3d ago

Characterizing Questioning Patterns and Student Engagement Through Contextual Analysis of Real-Time Classroom Interactions

The study analyzes 604 real‑time classroom poll questions from 47 live sessions, aligning each poll with lecture transcripts and attendance data. It finds that 89% of poll answers can be located in the lecture context, revealing that many polls serve attention‑checking functions only visible when contextualized. The majority of questions are lower‑order and fit into seven instructional functions, with student engagement high overall but uneven, and students often misjudge their own correctness.

By Rohit Sharma, Pavani Ayinampudi, Aditya B. M. V., Jinal Gupta, Prakash Hegade, Sakshi Sharma, Meenakshi V, SRS Iyengar
arXiv AI
Aug 18

Enhancing Science Classroom Discourse Analysis through Joint Multi-Task Learning for Reasoning-Component Classification

arXiv:2604. 21137v3 Announce Type: replace-cross Abstract: Analyzing the reasoning patterns of students in science classrooms is critical for understanding knowledge construction mechanism and improving instructional practice to maximize cognitive engagement, yet manual coding of classroom discourse at scale remains prohibitively labor-intensive.

By Jiho Noh, Mukhesh Raghava Katragadda, Raymond Carl, Soon Lee
arXiv Computation and Language
Sep 17

Which Demographics do LLMs Default to During Annotation?

The paper investigates which demographic attributes large language models (LLMs) default to when annotating text without explicit demographic cues. By comparing non‑demographic, placebo‑conditioned, and demographic‑conditioned prompts on politeness and offensiveness tasks in the POPQUORN dataset, the authors find that LLMs exhibit notable gender, race, and age influences in their annotations. This contrasts with earlier studies that reported no such effects, highlighting the importance of considering demographic bias in LLM‑based annotation workflows.

By Johannes Sch\"afer, Aidan Combs, Christopher Bagdon, Jiahui Li, Nadine Probol, Lynn Greschner, Sean Papay, Yarik Menchaca Resendiz, Aswathy Velutharambath, Amelie W\"uhrl, Sabine Weber, Roman Klinger