arXiv AI

Cryptanalytic Extraction of Isolated Bias-Free GLU Feed-Forward Blocks by Antipodal Separation

arXiv:2608. 06631v1 Announce Type: cross Abstract: Cryptanalytic extraction has been demonstrated for ReLU networks, for networks using componentwise activations such as GELU or SiLU, and for a Transformer's final projection matrix.

arXiv AI

Jun 9

Ghosted Layers: Unconstrained Activation Alignment for Recovering Layer-Pruned LLMs

arXiv:2605. 15491v2 Announce Type: replace-cross Abstract: Layer pruning removes entire Transformer decoder blocks from large language models, but introduces a mismatch between the hidden state received by the next surviving layer and the distribution it was trained to process, leading to significant performance degradation.

By Vincent-Daniel Yun, Junhyuk Jo, Sai Praneeth Karimireddy, Sunwoo Lee

llms efficiency safety

arXiv Machine Learning

Jul 16

Targeted Recovery of Weight-Space Mechanisms From Neural Networks

arXiv:2607. 13047v1 Announce Type: new Abstract: Parameter decomposition (PD) decomposes neural networks into interpretable computational components that faithfully reflect the original network's operations.

By Antoine Vigouroux, Lee Sharkey

llms

arXiv AI

Jun 11

ICA Lens: Interpreting Language Models Without Training Another Dictionary

arXiv:2606. 11722v1 Announce Type: cross Abstract: Finding interpretable directions in language-model representations is critical for understanding and controlling model behavior.

By Sida Liu, Feijiang Han

llms safety

arXiv AI

Jun 4

Recover-LoRA for Aggressive Quantization: Reclaiming Accuracy in 2-Bit Language Models via Low-Rank Adaptation with Knowledge Distillation on Synthetic Data

arXiv:2606. 04238v1 Announce Type: cross Abstract: Aggressive weight quantization to 2-bit precision offers substantial throughput and memory gains for large language model (LLM) inference, but typically incurs severe accuracy degradation.

By Devleena Das, Rajeev Patwari, Elliott Delaye, Ashish Sirasao

llms fine-tuning efficiency benchmarks

arXiv AI

Jun 2

DLLM-JEPA: Joint Embedding Predictive Architectures for Masked Diffusion Language Models

arXiv:2606. 00091v1 Announce Type: cross Abstract: Joint Embedding Predictive Architectures (JEPAs) have reshaped self-supervised representation learning in vision.

By Sangdae Nam

llms rag diffusion fine-tuning benchmarks

arXiv Machine Learning

Aug 10

The Sparsity Whisperer

arXiv:2608. 06630v1 Announce Type: new Abstract: Pruning reduces the inference cost of large language models, but existing criteria primarily preserve large activations or reconstruct layer outputs.

By Linghao Kong, Inimai Subramanian, Micah Adler, Dan Alistarh, Dan Gutfreund, Nir Shavit

llms efficiency multimodal

arXiv Machine Learning

Jun 25

Leaking Circuit Secrets: Gradient Leakage Attacks on Graph Neural Networks

arXiv:2606. 25589v1 Announce Type: new Abstract: As graph neural networks (GNNs) become standard tools for critical tasks in circuit design and analysis, their security and privacy risks require careful attention.

By Rupesh Raj Karn, Johann Knechtel, Ozgur Sinanoglu

efficiency benchmarks safety

arXiv AI

Jun 12

TWLA: Achieving Ternary Weights and Low-Bit Activations for LLMs via Post-Training Quantization

arXiv:2606. 13054v1 Announce Type: cross Abstract: Large language models (LLMs) exhibit exceptional general language processing capabilities, but their memory and compute costs hinder deployment.

By Zhixiong Zhao, Zukang Xu, Zhixuan Chen, Xing Hu, Zhe Jiang, Dawei Yang

llms efficiency

arXiv Machine Learning

Jul 31

WIDE: Boosting Adaptive LLM Inference via Token-level Dynamic Width Pruning

arXiv:2607. 28418v1 Announce Type: cross Abstract: Pruning is a promising approach for improving the efficiency of LLMs.

By Haozhe Hu, Hao Wu, Peiran Yin, Chao Han, Yunpu Ma, Xiaoyu Shen

llms nlp efficiency benchmarks

arXiv AI

5d ago

BGA: A noise-immune neural distillation framework for malicious signature extraction in high-entropy encrypted flows

arXiv:2608. 14126v1 Announce Type: cross Abstract: To mitigate attention dilution in high-entropy TLS 1.

By Sheng Hong, Yixuan Huang, Weiwei Jiang, Junyuan Zhang, Jiacheng Wang, Ruijian Jiao

llms diffusion efficiency benchmarks

arXiv AI

Jul 3

The Wiola Architecture for Efficient Small Language Models

arXiv:2607. 01394v1 Announce Type: new Abstract: We present Wiola, a fully original Small Language Model (SLM) architecture built from first principles, sharing no structural lineage with any existing model family including GPT, LLaMA, Mistral, or Falcon.

By Aryuemaan Kumar Chowdhury, Afreen Shaik, Yaparla Bhargavi, Brahma Kumar

llms

arXiv AI

Jul 20

Kolmogorov--Arnold Networks for Small Language Models

arXiv:2607. 15525v1 Announce Type: cross Abstract: Kolmogorov--Arnold Networks (KANs) replace fixed node activations with learned one-dimensional edge functions, offering an explicit interface for interpretation and a possible alternative to transformer feed-forward networks.

By Felippe Alves, Renato Vicente

llms efficiency benchmarks

Related stories