From MHA and GQA to MLA, sparse attention, and hybrid architectures
Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at Sebastian Raschka.
Linear Attention Hybrids, Text Diffusion, Code World Models, and Small Recursive Transformers
arXiv:2606. 05843v1 Announce Type: cross Abstract: While Multimodal Large Language Models (MLLMs) demonstrate remarkable proficiency on complex vision-language tasks, the mechanisms by which they extract query-relevant visual features from complex, noisy contexts remain opaque.
arXiv:2606. 15378v1 Announce Type: cross Abstract: Modern language models increasingly adopt hybrid architectures that combine full attention with efficient attention modules, such as sliding-window attention (SWA) and recurrent sequence mixers.
arXiv:2508. 16771v3 Announce Type: replace-cross Abstract: Code Language Models (CodeLLMs) learn token importance from data correlations, whereas human developers attend selectively to semantically salient code.
arXiv:2606. 15633v2 Announce Type: replace Abstract: Large Language Models (LLMs) have shown promise for reasoning over Text-Attributed Graphs (TAGs).
arXiv:2606. 15633v1 Announce Type: new Abstract: Large Language Models (LLMs) have shown promise for reasoning over Text-Attributed Graphs (TAGs).
