arXiv AI
Large-scale factor analysis shows machine intelligence is only partially interpretable
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Aug 18
Do Assessment Instruments Measure the Same Thing for Humans and LLMs? A Latent Structure Analysis
arXiv:2608. 15630v1 Announce Type: cross Abstract: The rapid development and growing deployment of large language models (LLMs) have made it increasingly important to understand their capabilities.
arXiv AI
Aug 19
Interpretable Humans, Alien LLMs: Expert Analysis of Latent Structures in Assessment Responses
The study examines whether the latent factors that explain large language model (LLM) performance correspond to human‑interpretable cognitive constructs. Using exploratory factor analysis on responses from humans and six LLMs in quantitative reasoning and chemistry, subject‑matter experts could interpret most human‑derived factors but struggled to ascribe meaning to LLM‑derived factors, especially in quantitative reasoning and only partially in chemistry. The results suggest that LLMs often rely on statistically opaque mechanisms distinct from human reasoning.
arXiv AI
Sep 1
Balance of Benchmarks: Semantic Density Reweighting for Benchmark Multiplicity and Task-Conditioned Evaluation
arXiv:2608.30044v1 Announce Type: new Abstract: Language models are commonly compared by averaging scores across a benchmark list with equal weight. Such lists grow through publication outside an exp...
arXiv AI
Jul 20
Latency-Response Theory Model: Evaluating Large Language Models via Response Accuracy and Chain-of-Thought Length
arXiv:2512. 07019v3 Announce Type: replace-cross Abstract: The proliferation of Large Language Models (LLMs) necessitates valid evaluation methods to provide guidance for both downstream applications and actionable future improvements.
arXiv AI
Jul 10
Persona Matters: Effects of Activation Steering on Short Answer Generation and Scoring
arXiv:2604. 07102v2 Announce Type: replace-cross Abstract: Activation-based steering enables inference-time personalization of large language models, but its effects in educational applications are not well understood.
arXiv AI
Jul 17
Simplicity Paradox: Debunking myths about prompting and datasets for LLM evaluation
arXiv:2607. 14109v1 Announce Type: cross Abstract: Probing the capabilities of Large Language Models (LLMs) and building robust solutions for Multiple-Choice Question Answering (MCQA) remain central challenges in natural language understanding.