Deep Learning over the Internet: Training Language Models Collaboratively

MLLM-LLaVA-FL: Multimodal Large Language Model Assisted Federated Learning

arXiv:2409. 06067v3 Announce Type: replace Abstract: Previous studies on federated learning (FL) often encounter performance degradation due to data heterogeneity among different clients.

The State Of LLMs 2025: Progress, Problems, and Predictions

A 2025 review of large language models, from DeepSeek R1 and RLVR to inference-time scaling, benchmarks, architectures, and predictions for 2026.

Projector Is All You Train

arXiv:2608. 19726v1 Announce Type: cross Abstract: The typical training process of a multimodal large language model (MLLM) involves adapting both the language model backbone and the projector between the backbone and a modality-specific encoder.

Language Model Networks: Supervision-Efficient Learning through Dense Communication

arXiv:2505. 12741v3 Announce Type: replace Abstract: Language models are increasingly used not only as standalone predictors but also as components in larger inference systems, from test-time scaling to multi-agent collaboration.

Copy-Paste Language Models

arXiv:2610.19247v1 Announce Type: new Abstract: Transformers are intuitively considered to be efficient at copying from their inputs. We actually show that the ability of LLMs to copy in context can...

Automatic register identification for the open web using multilingual deep learning

arXiv:2406. 19892v5 Announce Type: replace Abstract: This article presents multilingual deep learning models for identifying web registers -- text varieties such as news reports and discussion forums -- across 16 languages.

Red-Teaming Large Language Models

SharedSAE: One Feature Dictionary Across Language Models

SharedSAE demonstrates that a single sparse autoencoder can replace multiple model‑specific SAEs by using a shared dictionary with model‑specific encoder‑decoder pairs. It preserves activation magnitudes, normalizes only selection scores, and supports single‑model inference via model dropout. Trained on four 1B‑scale language models, SharedSAE retains 96.6% of the mean explained variance of dedicated SAEs, shows higher cross‑model latent correlations, and allows efficient adaptation of new models to the shared latent space.

Cosmopedia: how to create large-scale synthetic data for pre-training Large Language Models

Evaluating large language models trained on code

OpenESM: Energy-Steered Models Are Scalable Foundation Learners

arXiv:2610.33005v1 Announce Type: new Abstract: Energy-based models have shown promising properties for domain-specific tasks but remain challenging in modeling general-purpose scenarios. To bridge t...

One Adapter Pair per Model: A Universal Activation Interface for Language Models

arXiv:2608. 09521v1 Announce Type: new Abstract: Activation-based tools are usually tied to one model's native hidden space, requiring probes, sparse autoencoders, and natural-language interpreters to be rebuilt or rediscovered for each new language model.