Inverse decision modeling infers latent properties of decision processes from observed behavior, but existing formulations rely primarily on action trajectories. In verbalized cognitive tasks, task execution also produces response dynamics that action-only formulations leave unmodeled, such as verbal production, interaction, and hesitation.
arXiv:2608. 05224v1 Announce Type: new Abstract: Large language models fine-tuned on human behavioural data have emerged as general-purpose cognitive proxies, but the scale this requires, and whether these models process task structure or exploit statistical shortcuts, remain open questions.
The paper introduces Process-aware Language Cognitive Diagnosis (PLCD), a framework that replaces traditional ID-based embeddings in Cognitive Diagnosis Models with language-derived structures and response records. PLCD employs large language models to build concept schemas and cognitive process graphs, and uses a Language-to-Cognition Mapper with DA-MoE experts and contrastive learning to map textual evidence into a unified cognitive space. Experiments demonstrate that PLCD outperforms conventional baselines in student performance prediction and shows strong cognitive transfer, improving cold-start robustness and cognitive grounding.
The paper introduces a method that represents language models as log‑likelihood vectors over prompt‑response pairs, enabling the construction of model maps that compare conditional distributions. Squared Euclidean distances in this vector space approximate KL divergence, and experiments show that these maps reveal global structure related to model attributes and task performance. The approach also captures systematic shifts from prompt changes, supports additive compositionality for predicting downstream scores, and offers PMI vectors to mitigate unconditional distribution effects, thereby aiding analysis and prediction of input‑dependent behavior.
arXiv:2607. 29181v1 Announce Type: cross Abstract: Agentic assistants capable of proactive, personalized interactions require structured models of user intent and workflow.
arXiv:2606.03371v4 Announce Type: replace Abstract: Reliable proactive agents must choose an action and judge whether current evidence is sufficient to act. We study retail service from sparse third-...
arXiv:2608.13387v3 Announce Type: replace Abstract: On-policy distillation (OPD) supervises a student language model on trajectories sampled from its current policy, but assigns equal credit to respo...
The study evaluates how large language models (LLMs) interpret verbal probability expressions by mapping words to numbers and testing consistency across 19 models. Results show that LLMs largely mirror human benchmarks—preserving word order, recovering key anchor points, and reflecting the high variance of the term "possible"—but they exhibit a systematic upward bias for negative expressions like "unlikely" and "improbable." Explanation elicitation reduces within‑model variance but increases divergence between models, while a bidirectional roundtrip test reveals that leading models maintain coherent internal representations.
arXiv:2605. 23055v2 Announce Type: replace-cross Abstract: Frontier language models sometimes recognize that they are being evaluated and adjust their behavior, undermining validity of benchmark results.
Large language models increasingly produce and interpret verbal probability expressions, yet whether these expressions carry consistent meaning across models (or match human perceptions of uncertainty...
arXiv:2606. 00467v1 Announce Type: cross Abstract: Large Language Models (LLMs) are increasingly used for zero-shot annotation and LLM-as-a-judge tasks, yet their reliability hinges on how model-internalized priors interact with user-provided instructions.
On-policy distillation (OPD) supervises a student language model on trajectories sampled from its current policy, but assigns equal credit to response tokens with unequal supervision value. Selective OPD addresses this limitation by allocating supervision non-uniformly across response tokens according to their estimated training value.