arXiv AI

A Comparative Study of Student Perspectives on Technical Writing Feedback Quality: Evaluating LLMs, SLMs, and Humans in Computer Science Topics

arXiv:2601. 11541v2 Announce Type: replace-cross Abstract: To address the scalability of feedback in computer science while mitigating the privacy and cost limitations of commercial Large Language Models (LLMs), this study evaluates a locally hosted Small Language Model (SLM).

arXiv Computation and Language
Sep 3

Expos\'ia: Teaching and Assessment of Academic Writing Skills for Research Project Proposals and Peer Feedback

Exposía is the first public dataset linking academic writing and feedback in higher education, comprising student research project proposals, peer and instructor comments, and free-text reviews collected from a Computer Science course. It includes human assessment scores based on a fine‑grained, pedagogically‑grounded schema for both writing and feedback. The dataset is used to benchmark large language models on automated scoring of proposals and student reviews, revealing that different LLMs excel at each task and that closed‑source models outperform open‑weight ones, while a multi‑aspect prompting strategy proves most effective for classroom deployment.

By Dennis Zyska, Alla Rozovskaya, Ilia Kuznetsov, Iryna Gurevych
arXiv AI
Sep 24

Evaluating Feedback Focus and Pedagogical Adaptivity in LLM-Generated Feedback on Student Writing

The paper examines whether state‑of‑the‑art large language models (LLMs) produce feedback that aligns with expert teachers’ pedagogical practices, focusing on feedback type and adaptivity. Using a refined taxonomy of seven feedback focus types, the authors annotate and compare teacher and LLM‑generated feedback from three university writing courses, creating the FeedType benchmark. Their analysis shows that while most LLMs cover many feedback types, they do not match teachers’ distribution patterns or adaptive behavior across draft stages and student performance levels.

By Norah Almousa, Shayan Peyghambari Oskoui, Raquel Coelho, Gayle Rogers, Xiang Lorraine Li, Diane Litman
arXiv AI
Sep 7

Who Should Grade My Work? Student Perspectives on Transparent AI-Assisted Writing Assessment in Higher Education

The study explores how undergraduate computing students in Saudi Arabia perceive AI‑generated writing feedback when they are explicitly told that ChatGPT, not a human instructor, produced the score and comments. Through qualitative reflections, four themes emerged: students found the feedback useful for surface‑level revisions, recognized AI’s contextual and pedagogical limits, trusted the feedback conditionally—separating its utility from its authority—and reaffirmed the human instructor’s role as the ultimate grading authority. The findings highlight a clear distinction students make between feedback usefulness and evaluative authority, treating them as separate judgments rather than opposing ends of a single approval scale.

By Rayed AlGhamdi
arXiv AI
Sep 21

Self-Explanation Tutor for Active Study of CS1 Worked Examples

The paper presents ESSE, a self‑explanation tutor that uses a large language model to give immediate feedback on students’ line‑by‑line explanations of introductory programming worked examples. It evaluates the LLM’s judgments against a domain expert and a crowd of non‑experts, finding that the model is reliable enough to serve as the tutor’s assessment engine. In an introductory Java course, the tutor’s feedback encourages students to persist, improves the completeness and conceptual depth of their explanations, and shows evidence of learning.

By Arun-Balajiee Lekshmi-Narayanan, Mohammad Hassany, Kamil Akhuseyinoglu, Rully Hendrawan, Peter Brusilovsky
arXiv AI
Sep 25

Guardrails or Roadblocks? Effects of Pedagogical Style and Context Awareness in AI Teaching Assistants for Programming

The study examined how different designs of AI teaching assistants (AI TAs) affect students in an introductory programming course. Four AI TAs were compared based on pedagogical style (Socratic vs. Direct instruction) and context awareness (no context vs. full context). Results showed that the Socratic AI TA with full context received the lowest favorability ratings, had the highest interaction stress, the most external LLM use, and the lowest comprehension outcomes, though differences were not statistically significant.

By Madeleine Eastwood, Harshith Narne, Joseph Hilby, Paul Denny, Ashish Aggarwal, Amanpreet Kapoor