arXiv Machine Learning
2d ago
Identifiability and Order-Dimension Limits of In-Context Learning on Partial Orders
arXiv:2608. 14004v1 Announce Type: new Abstract: In-context learning is commonly formalized as inference from examples of a function.
Finite Certificates for In-Context Determinacy and a Threshold Theory of Emergence in Language Models
arXiv:2606. 07623v1 Announce Type: new Abstract: This paper develops a model-theoretic framework for verifying context-conditioned language-model behavior by replacing benchmark labels with finite semantic certificates.
Related stories
arXiv AI
2d ago
Split the Labor: Separating Evidence Interpretation from Decision Aggregation
arXiv:2608. 14509v1 Announce Type: new Abstract: Systems that ask a language model to reach a conclusion from many sources usually concatenate them into one prompt.
arXiv AI
Jul 10
Understanding Axes of Difficulty For Long Context Tasks Via PredicateLongBench
arXiv:2607. 08284v1 Announce Type: new Abstract: Large language models (LLMs) have demonstrated rapidly improving long-context capabilities, prompting a wave of benchmarks designed to evaluate them.
arXiv Machine Learning
1d ago
Cross-Entropy Risk Estimation for Language Models: Inconsistency Must Be Dense, and the Holdout Method Is No Exception
arXiv:2608. 15798v1 Announce Type: new Abstract: Language models are compared by their held-out per-token cross-entropy risk---the quantity scaling laws are fitted to.
Hugging Face Trending Papers
Aug 3
Measuring in-context algorithmic reasoning in language models against an exact Bayes-optimal standard
Whether large language models perform genuine algorithmic reasoning or mere pattern completion is hard to test, because most benchmarks lack a ground truth for correct inductive inference. We introduce F-ICL, an in-context-learning benchmark that supplies one exactly.
arXiv AI
Jul 13
NL-PAC: Specification Ambiguity and Certified Minimax Risk Floors in LLM-Mediated Supervision
arXiv:2607. 08961v1 Announce Type: cross Abstract: Large language models increasingly provide labels, evaluations, and feedback for tasks specified in natural language.
arXiv AI
5d ago
Large Language Models Can Follow Instructions, But Not Many at Once: Phase Transitions in Compositional Constraint Satisfaction
arXiv:2608. 12426v1 Announce Type: new Abstract: Large language models are increasingly deployed in settings that require simultaneous adherence to multiple explicit constraints - reasoning structure, safety boundaries, output schemas.
arXiv AI
Aug 5
The production of meaning in the processing of natural language
arXiv:2603. 20381v2 Announce Type: replace-cross Abstract: Understanding the fundamental mechanisms governing the production of meaning in the processing of natural language is critical for designing safe, thoughtful, engaging, and empowering human-agent interactions.
arXiv AI
Jun 16
The Faithfulness Gap: Certifying Semantic Equivalence Between Natural-Language and Formal Mathematical Statements
arXiv:2606. 16541v1 Announce Type: new Abstract: Autoformalization, translating natural-language mathematics into formal proof assistants, is bottlenecked not by translation fluency but by \emph{faithfulness}: a formal statement can typecheck and be provable, yet still encode a different theorem than the source intended.
arXiv AI
Jul 17
Information-Theoretic Limits of Reliability and Scaling in Language Models
arXiv:2607. 14112v1 Announce Type: cross Abstract: Large language models (LLMs) are evaluated as though perfect reliability is achievable for any task given sufficient scale.
arXiv AI
Jul 22
The Information Shadow: Measuring Structural Limits on What Language Models Can Learn
arXiv:2607. 18305v1 Announce Type: cross Abstract: Some limits on what language models know are not gaps in data coverage but structural properties of learning from text.
arXiv Machine Learning
Aug 4
Measuring in-context algorithmic reasoning in language models against an exact Bayes-optimal standard
arXiv:2608. 01575v1 Announce Type: new Abstract: Whether large language models perform genuine algorithmic reasoning or mere pattern completion is hard to test, because most benchmarks lack a ground truth for correct inductive inference.