Structured data exists in many forms (tables, knowledge graphs, charts, and time series), and converting it into text may involve different generation tasks. However, most prior work on data-to-text (...
The paper introduces Adaptive Dual-Teacher Distillation with Subnetwork Rectification (DDSR), a method for black-box domain adaptation that fuses predictions from a black-box source model and a vision‑language model to generate reliable pseudo‑labels for the target domain. DDSR uses adaptive prediction fusion, subnetwork regularization to prevent overfitting, iterative refinement of pseudo‑labels and prompts, and class‑wise prototypes for self‑training. Experiments on several benchmark datasets show that DDSR outperforms existing state‑of‑the‑art approaches, even those that have access to source data or source model parameters.
arXiv:2607. 11898v1 Announce Type: cross Abstract: Large-scale text corpora have become a quiet bottleneck in modern NLP, not just in storage, but in the accumulated cost of training, fine-tuning, and continual learning.
arXiv:2509. 09960v2 Announce Type: replace-cross Abstract: Synthetic tabular data generation is increasingly essential in machine learning, supporting downstream applications when real-world, high-quality tabular data is insufficient.
arXiv:2605. 15532v3 Announce Type: replace-cross Abstract: Distillation enables compact Vision-Language Models (VLMs) to obtain strong reasoning capabilities, yet the prompts driving this process are typically chosen via simple heuristics or aggregated from off-the-shelf datasets.
arXiv:2603. 07523v3 Announce Type: replace Abstract: Transferring knowledge by fine-tuning large-scale pre-trained networks has become a standard paradigm for downstream tasks, yet the knowledge of a pre-trained model is tightly coupled with monolithic architecture, which restricts flexible reuse across models of varying scales.
The paper introduces Adaptive Mutual Distillation (AMD), a post‑training framework that jointly trains two large language models using different task‑balancing strategies. AMD evaluates and selects distillation weight adjustments via short training probes and task‑wise validation, leading to models that outperform supervised fine‑tuning baselines on six benchmarks across three backbones. Merging the two AMD models further improves performance, surpassing multi‑task fine‑tuning by an average of 2.91 points.
The paper introduces SynPro, a synthetic data generation framework that augments limited organic text for large language model pretraining by applying rephrasing and reformatting operations. SynPro’s generators are optimized with reinforcement learning rewards for quality, faithfulness, and data influence, and are updated continuously as training plateaus. Experiments on 400M, 1.1B, and 2B models show that SynPro can unlock 3.4–5.2× the effective tokens of standard repetition, even outperforming a non‑data‑bound oracle at larger scales.
arXiv:2607. 17467v1 Announce Type: cross Abstract: Few-shot Test-Time Domain Adaptation (FSTT-DA) seeks to adapt models to novel domains using only a handful of unlabeled target samples.
arXiv:2609.37891v1 Announce Type: cross Abstract: Current pre-training datasets are derived from web crawls, with all their issues, and were not designed to support mid- and post-training pipelines--...
arXiv:2606. 00147v1 Announce Type: cross Abstract: Domain-specific supervised fine-tuning (SFT) often improves in-domain performance at the cost of degrading a model's general capabilities.
arXiv:2607. 07179v1 Announce Type: cross Abstract: Document Visual Question Answering (DocVQA) presents a complex multimodal challenge, requiring models to exploit visual, textual, and layout information from documents.