arXiv:2601. 05106v5 Announce Type: replace Abstract: Large language models (LLMs) exhibit strengths across diverse domains.
Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.
arXiv:2608.30158v1 Announce Type: cross Abstract: Supervised fine-tuning (SFT) is the de facto standard for adapting large language models (LLMs) to target domains, but it often degrades the model's...
Leveraging multiple specialized LLMs can combine complementary strengths, but existing approaches trade adaptability for stability: routing commits prematurely, heuristic ensembling depends on fragile proxies, and parameter merging introduces interference. We propose DLLG (Dynamic Logit-Level Gating), a dynamic logit-level ensembling framework that learns token-level expert fusion from sparse response-level supervision.
Supervised fine-tuning (SFT) is the de facto standard for adapting large language models (LLMs) to target domains, but it often degrades the model's general capabilities, a phenomenon known as catastr...
The paper introduces two token-error supervision methods for sparse mixture-of-experts (MoE) large language models, aligning routing affinities with token-level cross-entropy loss. The first method predicts an error score per expert and uses it to adjust affinities before top‑K selection, while the second directly aligns router affinities to the model’s objective without extra heads or inference changes. Experiments on Granite and ARC‑Challenge show accuracy gains of about 2.3–2.94 percentage points over a parameter‑matched baseline, preserving the native sparse execution budget.
arXiv:2606. 06098v1 Announce Type: cross Abstract: Foundational Large Language Models (LLMs) demonstrate proficiency on a wide range of general tasks, and achieve remarkable results on various specialized tasks via domain-expert LLMs.
arXiv:2608. 06819v1 Announce Type: cross Abstract: Token-level collaboration allows a large language model (LLM) to assist a small language model (SLM) when their predictions diverge.