arXiv AI
Aug 7
Subliminal Learning is Non-Semantic Distillation
arXiv:2608. 05734v1 Announce Type: new Abstract: Subliminal Learning (SL) is a surprising type of generalization displayed by modern language models.
Subliminal Learning Is Steering Vector Distillation
arXiv:2606. 00995v1 Announce Type: new Abstract: Subliminal learning refers to a student language model acquiring a teacher's traits (e.
Related stories
Hugging Face Trending Papers
Aug 6
Subliminal Learning is Non-Semantic Distillation
Subliminal Learning (SL) is a surprising type of generalization displayed by modern language models. It allows the transfer of a bias or behavior from a teacher model to a student by distilling from seemingly unrelated or random synthetic data from the teacher.
arXiv AI
Jul 13
GrAInS: Gradient-based Attribution for Inference-Time Steering of LLMs and VLMs
arXiv:2507. 18043v2 Announce Type: replace-cross Abstract: Inference-time steering methods offer a lightweight alternative to fine-tuning large language models (LLMs) and vision-language models (VLMs) by modifying internal activations at test time without updating model weights.
arXiv AI
Aug 3
On the Generalization of Steering Vectors for Chain-of-Thought Faithfulness
arXiv:2607. 29062v1 Announce Type: new Abstract: Model capabilities have improved in large part due to scaling chain of thought.
arXiv AI
Aug 11
DeltaPrompts: Escaping the Zero-Delta Trap in Multimodal Distillation
arXiv:2605. 15532v3 Announce Type: replace-cross Abstract: Distillation enables compact Vision-Language Models (VLMs) to obtain strong reasoning capabilities, yet the prompts driving this process are typically chosen via simple heuristics or aggregated from off-the-shelf datasets.
arXiv AI
Jul 29
Minimizing Targeted Activations: Input-Only Suppression of Evaluation-Awareness Latents in Large Language Models
arXiv:2607. 25907v1 Announce Type: cross Abstract: Activation steering controls model behavior by editing internal activations at inference time.
arXiv AI
Aug 5
When Teachers Mislead: Spurious-Signal-Aware On-Policy Distillation
arXiv:2608. 03632v1 Announce Type: new Abstract: On-Policy distillation (OPD) transfers teacher capabilities by supervising student-sampled trajectories with dense token-level teacher signals.
arXiv AI
Jun 2
Subliminal Learning is a LoRA Artifact
arXiv:2606. 00831v1 Announce Type: new Abstract: Subliminal learning is a phenomenon where language models can transmit behavioral traits to other models through seemingly innocuous data (Cloud et al.
arXiv Machine Learning
Jul 14
Diagnosing and Mitigating Thinking Collapse in On-Policy Self-Distillation
arXiv:2607. 10805v1 Announce Type: cross Abstract: On-Policy Self-Distillation (OPSD) has emerged as a crucial paradigm for enhancing and aligning Large Language Models (LLMs).
Hugging Face Trending Papers
Aug 4
When Teachers Mislead: Spurious-Signal-Aware On-Policy Distillation
On-Policy distillation (OPD) transfers teacher capabilities by supervising student-sampled trajectories with dense token-level teacher signals. Recent selective OPD methods improve this process by prioritizing signals that are confident, informative, or learnable.
arXiv AI
Jun 2
ASKD-Whisper: Adaptive Self-knowledge Distillation for Efficient and Low-Latency Automatic Speech Recognition
arXiv:2601. 19919v2 Announce Type: replace-cross Abstract: Knowledge distillation (KD) is one of the most effective paradigms for compressing large-scale foundation models into deployable architectures.
arXiv AI
Aug 11
The Transformer Revolution, Part 1: Dynamic Processing through Output-Weight Interconnections
arXiv:2608. 03921v2 Announce Type: replace Abstract: This paper offers a new interpretation of the Transformer during inference.