The Flow has not summarised this story yet — read it at arXiv Machine Learning.
The paper introduces semanticfa, an R package that performs semantic factor analysis on psychometric scales by converting items into language‑model embeddings and factoring their similarities. It demonstrates the method on the 50‑item International Personality Item Pool Big‑Five markers, compares the semantic factors to a large response‑based factor analysis, and shows how the package can interpret factors, flag redundant items, build short forms, and screen new items. The authors propose semantic‑behavioral coherence as a measurable property of a scale, noting that semantic factor analysis complements but does not replace traditional response‑based factor analysis.
The paper introduces a dual‑dimensional framework called Automated Item Similarity Analysis (AISA) that uses Large Language Models to assess incidental content similarity in large‑scale assessments. It combines Structured Decomposition and Semantic Relatedness to capture both structural and semantic nuances that traditional metrics miss. Psychometric validation shows that LLM‑derived metrics better align with construct‑irrelevant local dependence and produce more coherent item groupings, and simulations in Computerized Adaptive Testing demonstrate improved estimation stability and reduced bias with minimal efficiency loss.
The paper proposes a model-based evaluation framework that merges multidimensional item response theory (IRT) with question context embeddings to predict large language model (LLM) performance on unseen questions. By representing LLMs with latent capability profiles and incorporating question content to inform item characteristics, the approach improves prediction accuracy over model-free baselines in within-scenario settings and offers a richer description of capability variation than unidimensional models. However, the study also finds that this generalizability does not reliably extend to cross-scenario shifts, indicating a key limitation for broader application.
HyperPersona is a multi‑level hypergraph framework that models the hierarchical structure of text—document, sentence, and word—using hyperedges for documents and sentences and nodes for words. It employs a transformer‑based graph encoder to capture interactions across these linguistic layers, producing context‑sensitive, structurally grounded features for automatic personality prediction. Experiments on the Big Five personality dimensions demonstrate that HyperPersona, relying solely on text, outperforms state‑of‑the‑art baselines by effectively integrating multi‑level linguistic cues.
The paper introduces the Semantic‑Empirical Updating Framework (SEUF), which uses large language model paraphrases and embeddings aligned with hypothesized item‑factor assignments to create prespecified priors for Bayesian confirmatory factor analysis. In experiments with a 38‑item survey and two public surveys (FEVS and RIASEC), SEUF maintained semantic structure after wording controls and achieved comparable loading precision with 50 respondents to conventional CFA with 70–82 respondents, reducing required sample size by 29–39%. The approach yielded lower loading errors and better credible‑interval coverage, especially when respondent data were scarce, demonstrating the benefit of incorporating semantic priors in small‑sample measurement.
The paper introduces CHAP, a personalized generative retrieval framework that aligns query semantics with item representations through a hierarchical semantic alignment module and models user behavior using both discrete Semantic IDs and continuous representations. It also proposes a Residual Cascading Generation mechanism to reduce inference latency by limiting the Transformer decoder to a single pass. Experiments on multiple datasets and online A/B tests show that CHAP outperforms existing methods, demonstrating its practical value.