arXiv:2607. 20265v1 Announce Type: cross Abstract: Large-scale pretrained language models such as T5 and BERT have demonstrated strong capabilities for generating structured knowledge.
arXiv:2606. 24841v1 Announce Type: new Abstract: Prompt-based learning has emerged as a dominant paradigm in natural language processing.
The paper introduces T2D, a framework that prunes and adapts a target large language model into a draft model optimized for both inference latency and alignment with the target model’s outputs. By formulating draft model design as an optimization problem, T2D achieves up to 92% pruning while maintaining high speculative decoding performance across multiple target model families and domains. The resulting draft models can also be integrated with other speculative decoding techniques to yield additional gains.
arXiv:2602. 12996v2 Announce Type: replace-cross Abstract: Knowledge augmentation has significantly enhanced the performance of Large Language Models (LLMs) in knowledge-intensive tasks.
arXiv:2602. 15829v2 Announce Type: replace Abstract: The superficial alignment hypothesis (SAH) posits that large language models learn most of their knowledge during pre-training, and that post-training merely surfaces this knowledge.
arXiv:2606. 12117v1 Announce Type: cross Abstract: Benchmark scores often misrepresent a large language model's (LLM's) knowledge, because they rely, e.
arXiv:2509. 26169v2 Announce Type: replace Abstract: Alignment of large language models remains a central challenge in natural language processing.
arXiv:2607. 08399v1 Announce Type: cross Abstract: Large language models process prompts by propagating activations through dozens of layers before generating a response.
The paper introduces EvoPrompt, a framework for adapting vision‑language models to new tasks with limited data while preventing catastrophic forgetting. EvoPrompt uses a Modality‑Shared Prompt Projector to create hierarchical prompts and an evolutionary training strategy that separates low‑rank updates into directional and magnitude components, preserving learned semantic directions. Experiments show that EvoPrompt achieves state‑of‑the‑art few‑shot performance while maintaining the original zero‑shot capabilities of the pre‑trained models.
arXiv:2606. 05173v1 Announce Type: cross Abstract: Masked language modelling (MLM) has been the dominant pre-training objective for text encoders since BERT, yet it encourages representations that are strongly anchored to surface-form token identity rather than deeper semantic structure.
SKIM (Selective Knowledge-Informed Masking) is a pruning and distillation framework designed for large language model agents that operate under structured generation, where grammar constraints limit output to valid tokens. The method partitions token positions into context, decision, and format categories, using category-aware pruning to preserve parameters most relevant to decision positions and an on-policy distillation objective that aligns student and teacher on valid tokens with category-aware feature matching. Evaluations on tool-calling benchmarks and multi-step agent loops across various model scales and sparsity levels show that SKIM consistently outperforms baseline compression methods, especially in high-sparsity regimes where conventional techniques falter.
arXiv:2601. 03808v2 Announce Type: replace-cross Abstract: Large language models (LLMs) have achieved notable performance in code synthesis; however, data-aware augmentation remains a limiting factor, handled via heuristic design or brute-force approaches.
arXiv:2511. 14967v3 Announce Type: replace-cross Abstract: Large language models (LLMs) have shown great promise in generating structured diagrams from natural language descriptions, particularly Mermaid sequence diagrams for software engineering.