arXiv AI

From Memorization to Creation: Evaluating the Cognitive Depth of LLM-Generated Educational Questions

arXiv:2606. 18257v1 Announce Type: cross Abstract: While LLMs show promise in automating educational content creation, their ability to generate questions that stimulate higher-order thinking remains understudied.

arXiv AI

Jul 28

Beyond Direct Answering: Aligning Educational LLMs as Socratic Guides via Heuristic Reinforcement Learning

arXiv:2607. 22996v1 Announce Type: cross Abstract: Large language models (LLMs) deployed in educational settings often behave as direct answerers: they disclose target concepts in the opening turn instead of guiding students through progressive inquiry, as Socratic pedagogy prescribes.

By Xiaokun Wang, Siyu Song, Wentao Liu, Xiaodong Zou

llms reinforcement-learning safety

arXiv AI

Jun 15

Cross-Dataset Bloom Question Classification: Supervised Models and Prompted LLMs

arXiv:2606. 13684v1 Announce Type: cross Abstract: Automatic Bloom's taxonomy classification of assessment questions can substantially reduce instructor workload, but labeling is subjective and teacher-dependent.

By Abdolali Faraji, Mohammadreza Molavi, Zohreh Rasoulkhani, Mohammadreza Tavakoli, G\'abor Kismih\'ok

llms

arXiv AI

Jul 17

Simplicity Paradox: Debunking myths about prompting and datasets for LLM evaluation

arXiv:2607. 14109v1 Announce Type: cross Abstract: Probing the capabilities of Large Language Models (LLMs) and building robust solutions for Multiple-Choice Question Answering (MCQA) remain central challenges in natural language understanding.

By Inder Preet, Shuxin Lin, Dhaval Patel

llms nlp benchmarks

arXiv AI

Aug 5

Asking Questions the Right Way: A Multi-Agent Conversational System for Prompt Formulation in Complex Task Resolution

arXiv:2608. 01366v2 Announce Type: replace-cross Abstract: Large language models (LLMs) are integral to complex intellectual tasks, yet output quality remains constrained by user-provided prompts.

By B. Sankar, Pawni Yadav, Srinidhi Ranjini Girish, Amogh A. S

llms rag agents

arXiv Machine Learning

Jul 21

Explaining and Tuning Transformer-based LLMs in Arithmetic Tasks with Human Strategies

arXiv:2607. 17166v1 Announce Type: new Abstract: Transformer-based large language models (LLMs) continue to achieve state-of-the-art performance across various natural language processing tasks.

By Luyu Qiu, Jianing Li, Hwanhee Kim, Xiaoyong Wei, Yueyuan Zheng, Janet Hsiao, Lei Chen

llms nlp benchmarks safety

arXiv Machine Learning

Jun 8

Elmes*: Automated Construction of Fine-Grained Evaluation Rubrics for Large Language Models in Long-Tail Educational Scenarios

arXiv:2606. 06546v1 Announce Type: new Abstract: Evaluating large language models (LLMs) for education requires measuring how models teach, not only what they know.

By Tao Liu, Ye Lu, Ruohua Zhang, Siyu Song, Wentao Liu, Aimin Zhou, Hao Hao

llms agents benchmarks safety

arXiv AI

Jul 3

Automated grading of Linux/bash examinations using large language models: a four-level cognitive taxonomy approach

arXiv:2607. 02432v1 Announce Type: new Abstract: Scalable and reliable grading of command-line examinations remains a challenge in computing education, where rising enrolments make manual marking difficult and rule-based autograders cannot handle partial credit, equivalent solutions, or syntactic variation.

By Manuel Alonso-Carracedo, Ruben Fernandez-Boullon, Pedro Celard, Francisco J. Rodriguez-Martinez, Lorena Otero-Cerdeira

llms robotics safety

arXiv AI

Jul 3

DRIFTLENS: Measuring Memory-Induced Reasoning Drift in Personalized Language Models

arXiv:2607. 02374v1 Announce Type: new Abstract: Personalization changes what a model says to a user; we show that it can also change the reasoning trajectory used to justify the response.

By Xi Fang, Weijie Xu, Yingqiang Ge, Yuhui Xu, Stephanie Eckman, Chandan K. Reddy

llms reinforcement-learning

arXiv Machine Learning

Jul 2

CogTax: A Four-Level Cognitive Taxonomy for Command-Line Computing Education

arXiv:2607. 00140v1 Announce Type: cross Abstract: As computing education expands beyond traditional programming into operational domains such as systems administration and command-line environments, existing pedagogical frameworks struggle to capture a dimension that is critical in these contexts: the real-world consequences of learner actions.

By Manuel Alonso-Carracedo (Universidade de Vigo, Spain, IFCAE, Universidade de Vigo, Spain), Ruben Fernandez-Boullon (Universidade de Vigo, Spain, IFCAE, Universidade de Vigo, Spain), Pedro Celard (Universidade de Vigo, Spain, IFCAE, Universidade de Vigo, Spain), Francisco J. Rodriguez-Martinez (Universidade de Vigo, Spain, IFCAE, Universidade de Vigo, Spain), Lorena Otero-Cerdeira (Universidade de Vigo, Spain, IFCAE, Universidade de Vigo, Spain)

rag

arXiv AI

Jun 12

Creating and Evaluating K-12 GenAI Assessment Graders Through Context Engineering

arXiv:2606. 12422v1 Announce Type: cross Abstract: The integration of large language models (LLMs) into educational assessment represents a transformative shift in classroom grading practices.

By Zewei Tian, Alex Liu, Lief Esbenshade, Michael Xiao, Zachary Zhang, Yulia L\'apicus, Thomas Han, Kevin He, Min Sun

llms

arXiv AI

Jul 20

Mechanistic Interpretability of Cognitive Complexity in LLMs via Linear Probing using Bloom's Taxonomy

arXiv:2602. 17229v2 Announce Type: replace Abstract: The black-box nature of Large Language Models necessitates novel evaluation frameworks that transcend surface-level performance metrics.

By Bianca Raimondi, Maurizio Gabbrielli

llms safety

arXiv AI

5d ago

Assessment Design in the GenAI Era: The X1-X2-X3 Assessment Pattern for Testing Students' AI Literacy, Learning Outcomes, and Reflection

arXiv:2608. 12351v1 Announce Type: cross Abstract: Generative artificial intelligence (GenAI) has challenged the validity of unsupervised online assessment, especially in technical subjects where plausible answers can be produced with little effort.

By Riasat Islam (School of Electronic Engineering and Computer Science, Queen Mary University of London, London, United Kingdom), Thomas Roelleke (School of Electronic Engineering and Computer Science, Queen Mary University of London, London, United Kingdom)

llms

Related stories