arXiv Machine Learning

Text as an Index for Zero-Shot Cross-Domain Knowledge Tracing

The paper introduces TIZSKT, a zero‑shot cross‑domain knowledge tracing model that separates behavioral sequence modeling from domain‑local historical evidence construction. It uses text as an index via a frozen text encoder and unsupervised per‑domain projection to build question addresses, then aggregates a student’s past outcomes into a question‑specific historical correctness summary. Experiments on three public datasets show that TIZSKT outperforms baselines in mean AUC and accuracy, with ablations confirming the importance of text addressing and cross‑segment continuation.

arXiv AI
Sep 18

QVAC Genesis III: A Large-Scale, High-Quality Open Synthetic STEM Corpus for Efficient Language Model Pre-Training

QVAC Genesis III is a 191.43 B‑token synthetic STEM corpus covering 19 domains and multiple difficulty levels, created through a dual generation strategy that uses a weak edge‑scale student model to generate corrective explanations and contrastive reasoning. The authors evaluate the corpus with an LLM‑as‑a‑parser protocol and demonstrate that 1.7 B‑parameter models trained on QVAC Genesis III outperform those trained on Cosmopedia‑v2 and the Cosmo‑1B model on ARC, GPQA Diamond, and MMLU STEM benchmarks, achieving up to +28.57% improvement on ARC‑E and a 99.45% valid answer rate.

By Davide Vitabile, N. Ranjan, Akshay Nambiar, Kamal K. Gupta, Amril Nazir
arXiv Machine Learning
Sep 16

Does Continued Pretraining on a Learner Corpus Improve Automated Essay Scoring on English Proficiency Tests? Evidence from EFCAMDAT

The study examines whether domain‑adaptive continued pretraining (DAPT) on a learner‑writing corpus (EFCAMDAT) can enhance transformer‑based automated essay scoring (AES) for English proficiency tests. Researchers applied DAPT to BERT, RoBERTa, and DistilBERT and compared the adapted models with their original checkpoints on the FCE and IELTS datasets, evaluating both in‑domain scoring and few‑shot cross‑dataset transfer. Results show that full‑corpus DAPT yields mixed effects, while proficiency‑specific DAPT often outperforms full‑corpus DAPT and sometimes even the non‑adapted baseline, though benefits vary by proficiency composition and encoder architecture and do not consistently transfer across tests.

By Duy Anh Nguyen
arXiv Computation and Language
1d ago

From Lookups to Lessons: Turning Retrieved Knowledge into Parametric Memory

The paper introduces ResMem, a method that transforms retrieved knowledge into a parametric memory for large language models. By using the LLM as an offline teacher that reads retrieved text, ResMem learns an additive logit residual, avoiding large token storage and improving question‑answering performance. Experiments on Wikipedia and five QA datasets show significant storage savings and higher token F1 scores compared to k‑NN baselines.

By Changjiang Han, Bowei He, Ye Yuan, Weixu Zhang, Haolun Wu, Xue Liu, Jikun Kang
arXiv AI
Oct 27

TriageRAG: Failure-Aware Self-Training for Adapting Retrieval-Augmented Language Models to Specialized Domains

TriageRAG is a failure‑aware self‑training method for retrieval‑augmented language models that adapts them to specialized domains using only unlabeled text. It probes each self‑generated question with and without retrieved context, classifies the result into one of four categories (known, solved, context‑failure, retrieval‑failure), and routes each category to a tailored training signal, including a parametric route that teaches the reader to answer when retrieval misses. Experiments on 11 datasets across three domains and on CS‑Bench show that TriageRAG outperforms a budget‑matched SimRAG by 1.2–6.4 points and uniquely recovers closed‑book performance whenever retrieval harms the base model.

By Samarth Varshney, Ritu Yadav
arXiv AI
Aug 26

Incorporating Cognitive Load and Knowledge Transfer for Multi-Domain Knowledge Tracing

The paper introduces LT‑MKT, a new method for multi‑domain knowledge tracing that incorporates cognitive load and knowledge transfer. It constructs a multi‑domain hierarchical graph using textual information from questions and concepts, then explicitly models cross‑domain temporal and knowledge features to capture cognitive load effects. A knowledge transfer module further captures propagation of knowledge states within and across domains, leading to more accurate predictions of students’ future performance.

By Haotian Zhang, Shucun Wang, Jinze Wu, Liang Ding, Shuochen Liu, Zhenya Huang, Jing Sha, Shijin Wang, Qi Liu