arXiv AI

Using Evidence-Centered Design to Guide Prompt Engineering in Measurement: An Illustration with Automated Essay Scoring

arXiv AI
Aug 14

Assessment Design in the GenAI Era: The X1-X2-X3 Assessment Pattern for Testing Students' AI Literacy, Learning Outcomes, and Reflection

arXiv:2608. 12351v1 Announce Type: cross Abstract: Generative artificial intelligence (GenAI) has challenged the validity of unsupervised online assessment, especially in technical subjects where plausible answers can be produced with little effort.

By Riasat Islam (School of Electronic Engineering and Computer Science, Queen Mary University of London, London, United Kingdom), Thomas Roelleke (School of Electronic Engineering and Computer Science, Queen Mary University of London, London, United Kingdom)
Hugging Face Trending Papers
Jul 13

When the Target Domain Changes: AI-Mediated Construct Drift in High-Stakes English Language AssessmenW

High-stakes English proficiency tests treat standardized, unaided performance as evidence for score interpretations about academic English proficiency. This interpretation remains meaningful, but as target language use domains increasingly involve generative AI, the extrapolation from unaided test performance to academic communicative readiness becomes less self-evident.

arXiv AI
Sep 4

A Prompt-Engineering Approach to Develop Scalable, Flexible, and Real-Time Hybrid Micro-Level Personalization in a General Purpose AI Teaching Assistant

The paper introduces a prompt‑engineering framework that personalizes large language model (LLM) teaching assistants across disciplines by tailoring responses to six learner‑specific dimensions, creating 96 distinct learner profiles. It also analyzes student queries through Bloom’s Taxonomy to gauge cognitive complexity, encoding both learner attributes and cognitive assessments into structured prompts that condition the LLM without retraining. Experiments using NLP metrics and a small human study demonstrate that this approach yields perceptible differences in response style and structure, with statistical evidence linking specific learner attributes to measurable changes.

By Saptarshi Basu, Sandeep Kakar, Ashok Goel
arXiv AI
Aug 13

Making AI-Generated Feedback Matter: From Provision to Student Enactment

arXiv:2608. 11625v1 Announce Type: new Abstract: Feedback processes strongly influence student learning, yet their educational value depends on addressing two distinct challenges: providing high-quality, timely, and individualised feedback at scale, and supporting students to interpret, evaluate, and act on that feedback productively.

By Omar Alsaiari, Nilufar Baghaei, Jason M. Lodge, Dragan Ga\v{s}evi'c, Naomi Winstone, Hassan Khosravi
arXiv AI
Sep 7

A Human-in-the-Loop Framework for AI-Assisted Scoring in Large-Scale Writing Assessment

The paper introduces a human‑in‑the‑loop framework for AI‑assisted scoring of short written responses in a large‑scale national assessment. Using data from two recent test editions with about 5,000 student responses each, the authors validate that AI‑generated scores align moderately to highly with human raters across multiple rubric dimensions. The framework includes a correction workflow that flags cases needing human review, thereby reducing manual workload while maintaining assessment quality.

By Mar\'ia Eugenia Curi, Germ\'an Capdehourat, Isabel Amigo, Magdalena Romano, Rosana Serra, Adri\'an Silveira, Andr\'es Peri
Hugging Face Trending Papers
Aug 12

Making AI-Generated Feedback Matter: From Provision to Student Enactment

Feedback processes strongly influence student learning, yet their educational value depends on addressing two distinct challenges: providing high-quality, timely, and individualised feedback at scale, and supporting students to interpret, evaluate, and act on that feedback productively. Generative AI offers a credible means of addressing the provision challenge, but students' uptake of AI-generated feedback remains limited.

arXiv AI
Aug 17

Making AI-Generated Feedback Matter: A Large-Scale Study of Feedback Workflows and Student Enactment

arXiv:2608. 11625v2 Announce Type: replace Abstract: Feedback processes strongly influence student learning, yet their educational value depends on addressing two distinct challenges: providing high-quality, timely, and individualised feedback at scale, and supporting students to interpret, evaluate, and act on that feedback productively.

By Omar Alsaiari, Nilufar Baghaei, Jason M. Lodge, Dragan Ga\v{s}evi'c, Naomi Winstone, Hassan Khosravi