arXiv AI
"Act Like a 5th Grader" is Not Enough: Bounding Knowledge in LLM-Based User Simulators
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Aug 19
KnowSim: Evaluating Information Calibration in LLM Assistants with User Simulators that Learn
KnowSim introduces an evaluation framework that uses a user simulator with explicit knowledge states to assess how well large language models calibrate information to users. The simulator represents knowledge as a graph of Information Units with prerequisite relationships and updates these states based on learning theory. KnowSim computes Knowledge Gain, Delivery Calibration, and Cognitive Overload metrics, and its rankings align with human judgments, outperforming baseline simulators and revealing model performance differences across user knowledge levels.
arXiv AI
Jun 16
Edu-Theater: A Data-Efficient Agent Framework for Scalable Learner Behavior Simulation through Staging Roll-Call
arXiv:2606. 15225v1 Announce Type: cross Abstract: Large-scale learner-task interaction data are crucial for intelligent educational systems but are costly to collect and constrained by privacy and learner engagement.
arXiv AI
Jul 24
AI Assistants Overassist
arXiv:2607. 21306v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used as tutors and thought partners, helping users reason through problems.
arXiv Machine Learning
Aug 4
MemoryForge: Synthesize Lifelong Memory for Human-Like LLM Agents
arXiv:2608. 00007v1 Announce Type: cross Abstract: Equipping Large Language Models (LLMs) with human-like personas is crucial for agentic applications, such as role-play and user simulation.
arXiv AI
Aug 12
INSIDE the Student's Mind: Jointly Modeling Latent Reasoning and Action in LLM Student Simulators
arXiv:2608. 10492v1 Announce Type: new Abstract: Large Language Model (LLM)-based simulators often reproduce observable actions but fail to capture the underlying reasoning behind them.
arXiv Machine Learning
Sep 4
SWIM: Student Writing Simulation via Proficiency-Conditioned Generation
The paper introduces SWIM, a task that frames student writing simulation as proficiency‑conditioned essay generation. It evaluates prompting, supervised fine‑tuning, and reinforcement learning for aligning generated essays with student proficiency profiles, using automated essay scoring as a metric. Results show that prompting alone offers limited control, while supervised fine‑tuning and reinforcement learning significantly improve alignment across content, lexical, grammatical, and organizational traits, though low‑proficiency writing remains difficult to replicate.