arXiv AI
Generalization Is Stability, Not Accuracy: Multi-Axis Evaluation of LLMs
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
Hugging Face Trending Papers
Jul 14
The Illusion of Robustness: Aggregate Accuracy Hides Prediction Flips under Task-Irrelevant Context
As large language models (LLMs) grow more capable, they are increasingly deployed in context-rich settings where task inputs are often accompanied by long, partially irrelevant context. In a controlled setting, we find that state-of-the-art models often appear robust to task-irrelevant context at the aggregate level: prepending it to benchmark questions causes little change in overall accuracy.
arXiv AI
Sep 1
Emergent Misalignment Is Not Magical
arXiv:2608.29118v1 Announce Type: new Abstract: Fine-tuning large language models (LLMs) on narrowly harmful datasets can lead to misalignment broadly, a phenomenon known as emergent misalignment (EM...
arXiv AI
Jul 13
Contrastive Weak-to-strong Generalization
arXiv:2510. 07884v2 Announce Type: replace-cross Abstract: Weak-to-strong generalization provides a promising paradigm for scaling large language models (LLMs) by training stronger models on samples from aligned weaker ones, without requiring human feedback or explicit reward modeling.
arXiv AI
Aug 13
When the API Speaks the Wrong Language: Revisiting Post-Training for Multilingual Tool Use
arXiv:2608. 11715v1 Announce Type: cross Abstract: The reliability of Large Language Models (LLMs) for API calling degrades in multilingual settings.
arXiv Computation and Language
Sep 22
The Harder Text Embedding Benchmark (HTEB): Beyond One-dimensional Static Robustness
arXiv:2605.28190v2 Announce Type: replace Abstract: Embedding benchmarks like MTEB report a single score per model, implicitly treating robustness as a static, scalar property. We argue that embeddin...
arXiv AI
Jul 21
Prismatic Synthesis: Gradient-based Data Diversification Boosts Generalization in LLM Reasoning
arXiv:2505. 20161v2 Announce Type: replace-cross Abstract: Effective generalization in language models depends critically on the diversity of their training data.