arXiv Machine Learning
Jun 10
CodeAlchemy: Synthetic Code Rewriting at Scale
arXiv:2606. 10087v1 Announce Type: cross Abstract: Pre-training on raw code teaches syntax but provides sparse signal for diverse real-world task formats.
QAQ: Bidirectional Semantic Coherence for Selecting High-Quality Synthetic Code Instructions
Related stories
arXiv AI
Oct 7
When Plans Change Answers: Formalizing Cost-Accuracy Optimization for Semantic Queries
arXiv:2610.08089v1 Announce Type: cross Abstract: In semantic query engines, predicates are evaluated by machine-learned models, and the choice of a query plan affects not only the cost of a query bu...
arXiv Machine Learning
Jul 21
Efficient Sequential Evaluation of Large Language Models
arXiv:2607. 17409v1 Announce Type: cross Abstract: We study the problem of sequentially evaluating a new large language model (LLM) on a fixed question set using historical performance data from prior LLMs.
arXiv AI
6d ago
An Embarrassingly Simple Data Selection Strategy for Million-Scale Multi-Modal Instruction-Tuning
The paper introduces SSR, a simple yet effective data selection strategy for million‑scale multi‑modal instruction‑tuning. SSR uses interaction styles as a proxy for structural diversity and decomposes data quality into capability‑specific scores, forming a rich profile for each sample. By casting selection as submodular coverage maximization, SSR consistently outperforms random sampling across two architecture stacks, multiple data pools, and 14 benchmarks, achieving 99.1% of full‑data performance with only 30% of a 2.6M pool.
arXiv Machine Learning
Jul 24
DataPrep-Bench: Benchmarking LLMs as Training Data Preparators
arXiv:2607. 20465v1 Announce Type: new Abstract: The quality of training data fundamentally determines the capabilities of large language models (LLMs), yet no unified benchmark exists to measure how well LLMs, agents, and data-centric workflows actually prepare training data end to end.
arXiv AI
Jul 28
Loong: Synthesize Long Chain-of-Thoughts at Scale through Verifiers
arXiv:2509. 03059v2 Announce Type: replace-cross Abstract: Recent advances in Large Language Models (LLMs) have shown that their reasoning capabilities can be significantly improved through Reinforcement Learning with Verifiable Reward (RLVR), particularly in domains like mathematics and programming, where ground-truth correctness can be automatically evaluated.
Hugging Face Trending Papers
Oct 6
When Plans Change Answers: Formalizing Cost-Accuracy Optimization for Semantic Queries
In semantic query engines, predicates are evaluated by machine-learned models, and the choice of a query plan affects not only the cost of a query but also its result. Existing systems either apply a...
arXiv Machine Learning
Sep 14
CoHyDE: Iterative Co-Training of LLM Rewriter & Dense Encoder for Tool Retrieval
CoHyDE is an iterative co‑training framework that jointly trains a dense encoder and an LLM rewriter for tool retrieval from large API catalogs. The encoder is fine‑tuned with InfoNCE on catalog‑style hypothetical descriptions generated by the rewriter, while the rewriter is preference‑aligned via DPO against the encoder’s retrieval scores. On a 10k‑tool subset of ToolBench, three rounds of CoHyDE outperform the best single‑component baseline by 2.5 pp NDCG@5 on standard queries and 6.3 pp on vague queries, with the largest gains on the hardest vague tier.
arXiv AI
Aug 18
LLMs Get Smarter from Targeted Synthetic Multilingual Data
arXiv:2608. 15964v1 Announce Type: cross Abstract: Language-specific competency (LSC) is the phenomenon of a language model performing better or worse depending on the language of the prompt.
arXiv AI
Sep 3
SCX Router: Streaming Zero-Shot Model Selection with a Decoder-KV Classifier and a Real-World Task Ontology
The paper introduces SCX Router, a lightweight GLiClass-based model selector that assigns suitability scores to inference-time language models without autoregressive generation. It uses a 0.6B-parameter Qwen3 decoder with a shallow bidirectional scorer, preserving a text-only key–value cache across sessions and predicting task attributes such as type, difficulty, and expected output length. The authors build a comprehensive task ontology with 23 families, 115 types, and 1,173 synthetic examples, generating 150,000 verifier-scored tasks to train the router, which outperforms baseline models on LiveBench subsets with a top‑1 score of 0.707 versus 0.696 for the strongest fixed model.
arXiv Machine Learning
Jun 19
A Critical Look at Targeted Instruction Selection: Disentangling What Matters (and What Doesn't)
arXiv:2602. 14696v2 Announce Type: replace Abstract: Instruction fine-tuning of large language models (LLMs) often involves selecting a subset of instruction training data from a large candidate pool, using a small query set from the target task.
arXiv Computation and Language
Oct 8
SEER: Self-Enhancing Chain-of-Thought Compression for Reasoning Models
arXiv:2509.14093v3 Announce Type: replace-cross Abstract: Chain-of-Thought (CoT) prompting can substantially improve the reasoning ability of large language models (LLMs), but it often comes with hig...