arXiv Computation and Language

A Unified Framework to Elicit Structured Feedback for Interpretable Multi-Trait Essay Scoring

The paper introduces HiFTS, a unified autoregressive framework that generates hierarchical chain-of-thought (CoT) feedback before predicting trait-level and holistic scores for multi-trait automated essay scoring. HiFTS distills rubric-grounded CoT feedback from a teacher large language model and trains student models to jointly produce feedback and scores, employing Group Relative Policy Optimization to balance score agreement, calibration, feedback quality, and structural validity. The authors also present CFMS-34, a new Chinese multi-trait AES dataset, and demonstrate that HiFTS achieves strong scoring performance while producing coherent, rubric-aligned feedback on CFMS-34 and ASAP++.

arXiv Computation and Language
Sep 7

Trait-Aware Policy Optimization for Autoregressive Multi-Trait Essay Scoring

The paper introduces Trait-Aware Policy Optimization (TAPO), a post‑training framework for autoregressive models that score essays across multiple traits. TAPO decomposes rewards by sample and trait, integrating global consistency, trait accuracy, format validity, and inter‑trait dependencies, while enriching prompts with trait descriptions. Experiments on various backbone models show TAPO consistently outperforms supervised fine‑tuning and scalar‑reward baselines, proving its effectiveness and transferability for multi‑trait essay scoring.

By Zhengyang Wang, Sanwoo Lee, Jiaxin Wang, Chenxi Miao, Weikang Li, Yunfang Wu
Hugging Face Trending Papers
Jul 21

Beyond Score Prediction: LLM-Based Essay Scoring and Feedback Generation via Reinforcement Learning with Rubric Rewards

Large language models (LLMs) have been widely applied to automated essay scoring (AES) and automated feedback generation (AFG). However, existing studies rely primarily on prompt engineering or supervised fine-tuning, while systematic research on reinforcement learning (RL) post-training and automated evaluation of feedback quality remains limited.

arXiv AI
Aug 26

Learning to Grade Efficiently: A Bandit-Driven Prompt-Selection Framework for Low-Cost LLM Essay Scoring

The paper introduces a cost‑aware framework that treats each prompt type as an arm in a multi‑armed bandit controller, enabling adaptive selection of optimal prompting strategies during inference for automated essay scoring. Experiments on IELTS Writing Task 2 essays demonstrate that this bandit-driven approach achieves comparable scoring accuracy to exhaustive grid search while reducing LLM calls by 78.4%. The study also presents the first cost‑reliability learning curves for essay scoring, offering actionable insights for educational technology platforms balancing operational costs against assessment validity.

By Olga Manakina, Igor Bogdanov
arXiv AI
Sep 7

A Human-in-the-Loop Framework for AI-Assisted Scoring in Large-Scale Writing Assessment

The paper introduces a human‑in‑the‑loop framework for AI‑assisted scoring of short written responses in a large‑scale national assessment. Using data from two recent test editions with about 5,000 student responses each, the authors validate that AI‑generated scores align moderately to highly with human raters across multiple rubric dimensions. The framework includes a correction workflow that flags cases needing human review, thereby reducing manual workload while maintaining assessment quality.

By Mar\'ia Eugenia Curi, Germ\'an Capdehourat, Isabel Amigo, Magdalena Romano, Rosana Serra, Adri\'an Silveira, Andr\'es Peri
Hugging Face Trending Papers
Sep 4

A Human-in-the-Loop Framework for AI-Assisted Scoring in Large-Scale Writing Assessment

The paper introduces a human‑in‑the‑loop framework for AI‑assisted scoring of short written responses in a large‑scale national assessment. Using data from two recent test editions with about 5,000 responses each, the study validates that AI-generated scores align moderately to highly with human raters across multiple rubric dimensions. The framework also identifies when human review is most needed, allowing more efficient allocation of expert effort while maintaining assessment quality.