arXiv Computation and Language October 8, 2026 By Zhifan Sun, Sebastian Gombert, Jannik Lossjew, Tobias Wyrwich, Berrit Katharina Czinczel, David Bednorz, Marcus Kubsch, Knut Neumann, Hendrik Drachsler

Alice: A Large-Scale German Benchmark for Rubric-Based Multi-Dimensional Automatic Short Answer Scoring

Read the original on arXiv Computation and Language →

Alice is a new large-scale German benchmark for Automatic Short Answer Scoring that focuses on rubric-based evaluation. It contains three subtasks—learning performance (Alice‑LP), knowledge elements (Alice‑KE), and skills (Alice‑SK)—and is designed to assess how well students master underlying concepts and skills rather than just answer questions directly. Experiments show that large language models struggle with zero‑shot scoring of knowledge elements and skills, and that rubric text is especially helpful for Alice‑KE and Alice‑SK.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

arXiv Computation and Language

5d ago

Rubric Spans are Label Representations: Joint LLM Encoding for Short Answer Scoring

arXiv:2610.09660v1 Announce Type: new Abstract: Automatic Short Answer Scoring (ASAS) requires models that can score student responses against question-specific criteria while remaining efficient and...

By Zhifan Sun, Sebastian Gombert, Fabian Zehner, Leon Camus, Longwei Cong, Hendrik Drachsler

llms benchmarks

arXiv Machine Learning

Jun 8

Elmes*: Automated Construction of Fine-Grained Evaluation Rubrics for Large Language Models in Long-Tail Educational Scenarios

arXiv:2606. 06546v1 Announce Type: new Abstract: Evaluating large language models (LLMs) for education requires measuring how models teach, not only what they know.

By Tao Liu, Ye Lu, Ruohua Zhang, Siyu Song, Wentao Liu, Aimin Zhou, Hao Hao

llms agents benchmarks safety

arXiv AI

Aug 21

Inject, Align, Recover: Staged Post-Training for Retrieval-Free Document Knowledge Internalization

arXiv:2608. 20281v1 Announce Type: cross Abstract: Large language models often fail to answer questions about a bounded document collection when the source documents are not retrieved at inference time.

By Qian Kou, Xiaofeng Shi, Xiaosong Qiu, Hua Zhou

llms nlp fine-tuning benchmarks safety

arXiv AI

Oct 1

Instruction Retrieval at Inference Time for Small Language Models

arXiv:2510.13935v3 Announce Type: replace-cross Abstract: The facts a language model stores are tied to its parameter count, so small models that fit on edge devices fail on expert problems, which ne...

By Kenan Alkiek, David Jurgens, Vinod Vydiswaran

llms fine-tuning benchmarks

arXiv AI

Jul 29

Comparing RAG and GraphRAG for Page-Level Retrieval Question Answering on a Math Textbook

arXiv:2509. 16780v3 Announce Type: replace-cross Abstract: Large language models (LLMs) show promise as educational aids but often lack alignment with specific course materials.

By Eason Chen, Chuangji Li, Eric Li, Zimo Xiao, Jionghao Lin, Kenneth R. Koedinger

llms rag nlp safety

arXiv AI

Sep 30

SRJudge: Empowering Large Language Models with Selective Reasoning for Fine-Grained Knowledge Concept Tagging

arXiv:2609.36982v1 Announce Type: cross Abstract: Knowledge concept tagging aims to assign specific concept or topic labels to educational content, which is essential for both educators and learners...

By Zhiwei Yang, Jiahua Yang, Huiru Lin, Xing Chen, Quanlong Guan

llms reinforcement-learning fine-tuning efficiency benchmarks

One email a morning, machine-written

Related stories

Rubric Spans are Label Representations: Joint LLM Encoding for Short Answer Scoring

Elmes*: Automated Construction of Fine-Grained Evaluation Rubrics for Large Language Models in Long-Tail Educational Scenarios

Inject, Align, Recover: Staged Post-Training for Retrieval-Free Document Knowledge Internalization

Instruction Retrieval at Inference Time for Small Language Models

Comparing RAG and GraphRAG for Page-Level Retrieval Question Answering on a Math Textbook

SRJudge: Empowering Large Language Models with Selective Reasoning for Fine-Grained Knowledge Concept Tagging