← Back to all news
arXiv AI By Michael Lawrence Castanares, Princess Ventures, Allan Tan

Evaluation of pre-trained models for pedagogical assessment of novel AI-assisted educational questions

Read the original on arXiv AI →

The study evaluates how well pre‑trained models can classify the Bloom level of AI‑generated educational questions, a task that is crucial for ensuring pedagogical quality. Traditional machine‑learning models perform poorly on out‑of‑distribution data, whereas transformer and large‑language models achieve higher accuracy, especially after feature‑engineering techniques such as text splicing and appending learning objectives. Retraining the models yields the most significant performance gains across all datasets.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv AI
Sep 18

QVAC Genesis III: A Large-Scale, High-Quality Open Synthetic STEM Corpus for Efficient Language Model Pre-Training

QVAC Genesis III is a 191.43 B‑token synthetic STEM corpus covering 19 domains and multiple difficulty levels, created through a dual generation strategy that uses a weak edge‑scale student model to generate corrective explanations and contrastive reasoning. The authors evaluate the corpus with an LLM‑as‑a‑parser protocol and demonstrate that 1.7 B‑parameter models trained on QVAC Genesis III outperform those trained on Cosmopedia‑v2 and the Cosmo‑1B model on ARC, GPQA Diamond, and MMLU STEM benchmarks, achieving up to +28.57% improvement on ARC‑E and a 99.45% valid answer rate.

By Davide Vitabile, N. Ranjan, Akshay Nambiar, Kamal K. Gupta, Amril Nazir
llmsefficiencybenchmarks
More like this →
arXiv Machine Learning
Sep 16

Does Continued Pretraining on a Learner Corpus Improve Automated Essay Scoring on English Proficiency Tests? Evidence from EFCAMDAT

The study examines whether domain‑adaptive continued pretraining (DAPT) on a learner‑writing corpus (EFCAMDAT) can enhance transformer‑based automated essay scoring (AES) for English proficiency tests. Researchers applied DAPT to BERT, RoBERTa, and DistilBERT and compared the adapted models with their original checkpoints on the FCE and IELTS datasets, evaluating both in‑domain scoring and few‑shot cross‑dataset transfer. Results show that full‑corpus DAPT yields mixed effects, while proficiency‑specific DAPT often outperforms full‑corpus DAPT and sometimes even the non‑adapted baseline, though benefits vary by proficiency composition and encoder architecture and do not consistently transfer across tests.

By Duy Anh Nguyen
llms
More like this →