arXiv Machine Learning
From the Zoo to the BULLPEN: Sample-Efficient LLM Profiling via Multi-Channel Model Embeddings
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv Machine Learning
Oct 8
Latent Performance Profiling of Large Language Models
arXiv:2605.30018v3 Announce Type: replace-cross Abstract: Large language models (LLMs) frequently achieve impressive scores on standardized benchmarks, yet accuracy alone offers a limited view of the...
arXiv AI
Sep 25
PROOF: Profiling Reliability of Object-Level Facts in Large Language Models
PROOF is a benchmark that profiles the reliability of object-level facts in instruction-tuned language models by converting a frozen Wikidata snapshot into 18,486 English multiple-choice questions grounded in 11,779 semantic facts across 101 classes, 392 properties, and 14 domains. Each question includes an explicit "I don't know" option, a "No correct option" control, and nine controlled formulations, with 1,849 questions designed as no-correct-option traps. The study evaluates 18 open-weight model deployments on 166,374 prompts, revealing wide variability in factual accuracy, sensitivity to wording changes, and the impact of decoder perturbations.
Hugging Face Trending Papers
Oct 22
ULME: Predictive Coupling for Mutually Reinforcing Embedding and Language Modeling
ULME introduces a post‑training framework that unifies text embedding and generation within a single causal decoder‑only LLM by inserting learnable slots for semantic and reasoning embeddings. It trains three objectives—contrastive, language‑modeling, and stop‑gradient predictive—so that generation actively shapes the embedding space. Across Qwen2.5 and Qwen3 models (0.5B–8B), ULME improves MTEB scores by 3.3–4.1 points, tightens BRIGHT and AdvBench‑IR, and matches generation‑only fine‑tunes on standard accuracy benchmarks.
arXiv Computation and Language
Oct 23
ULME: Predictive Coupling for Mutually Reinforcing Embedding and Language Modeling
ULME is a post‑training framework that unifies text embedding and language modeling within a single causal decoder‑only LLM by inserting learnable slots for semantic and reasoning embeddings. It trains three objectives—contrastive loss for semantic alignment, language‑modeling loss for answer generation, and a stop‑gradient predictive loss linking reasoning to the answer—so that generation actively shapes the embedding. Across Qwen2.5 and Qwen3 models (0.5B–8B), ULME improves MTEB scores by 3.3–4.1 points, tightens BRIGHT and AdvBench‑IR, and matches generation‑only fine‑tunes on standard accuracy benchmarks.
arXiv Machine Learning
Jun 2
Test-Time Compute for Frozen Embedding Models through Agentic Program Search
arXiv:2605. 11374v5 Announce Type: replace Abstract: Test-time compute is widely believed to benefit only large reasoning models, leaving small models with nothing to gain.
arXiv AI
Jul 31
ARC-Encoder: learning compressed text representations for large language models
arXiv:2510. 20535v2 Announce Type: replace-cross Abstract: Recent techniques such as retrieval-augmented generation or chain-of-thought reasoning have led to longer contexts and increased inference costs.