arXiv AI
Jun 16
Nightjar: Dynamic Adaptive Speculative Decoding for Large Language Models Serving
arXiv:2512. 22420v5 Announce Type: replace-cross Abstract: Speculative decoding (SD) accelerates LLM inference by verifying draft tokens in parallel.
Leaky Language Models: Stealing Architecture and Inference Optimizations via Per-Token Timing
arXiv:2607. 20723v1 Announce Type: cross Abstract: This work presents LeakyLMs, a set of attacks that leak proprietary model, architecture, and deployment information from production language models.
Related stories
arXiv AI
Jul 3
Black-Box Inference of LLM Architectural Properties with Restrictive API Access
arXiv:2607. 01313v1 Announce Type: cross Abstract: In practice, most commercial LLM providers do not publicly release details of underlying LLM architectures.
Hugging Face Trending Papers
Jul 27
DraftExpert: Expansion-Aware Self-Speculative Decoding for End-Device MoE Inference
Large Mixture-of-Experts (MoE) language models are attractive for end-device deployment because only a small subset of experts is active per token, but their routed expert weights often exceed accelerator memory. We target latency-critical single-user settings where routed experts are staged on demand from CPU memory to a GPU or from Flash to a mobile NPU.
arXiv AI
Jun 17
Prefill/Decode-Aware Evaluation of LLM Inference on Emerging AI Accelerators
arXiv:2606. 17104v1 Announce Type: cross Abstract: As large language models (LLMs) are increasingly deployed in latency- and cost-sensitive settings, inference efficiency has become a central systems challenge.
arXiv AI
Jul 28
DraftExpert: Expansion-Aware Self-Speculative Decoding for End-Device MoE Inference
arXiv:2607. 24434v1 Announce Type: cross Abstract: Large Mixture-of-Experts (MoE) language models are attractive for end-device deployment because only a small subset of experts is active per token, but their routed expert weights often exceed accelerator memory.
arXiv AI
Jul 7
DSpark: Confidence-Scheduled Speculative Decoding with Semi-Autoregressive Generation
arXiv:2607. 05147v1 Announce Type: new Abstract: Speculative decoding accelerates Large Language Model (LLM) inference by decoupling draft generation from target verification.
arXiv AI
Jul 21
Lossless but Not Free: An Empirical Anatomy of Speculative Decoding on Consumer Hardware
arXiv:2607. 17283v1 Announce Type: new Abstract: Single-stream autoregressive decoding of large language models is bound by memory bandwidth: each generated token requires one full forward pass through the target model, and successive passes cannot be parallelized.
arXiv AI
Jun 2
Hybrid Verified Decoding: Learning to Allocate Verification in Speculative Decoding
arXiv:2606. 01019v1 Announce Type: cross Abstract: Large Language Model (LLM) generation remains expensive because autoregressive decoding calls the model once for each new token.
arXiv AI
Jul 24
Profiling Lightweight Large Language Models
arXiv:2607. 20806v1 Announce Type: new Abstract: Lightweight large language models (LLMs) are increasingly being deployed locally on personal computers and are expected to play a growing role in resource-constrained edge and mobile environments.
arXiv Machine Learning
Jul 27
Adversarial Prompts for Acceptance Collapse in Speculative Decoding
arXiv:2607. 21804v1 Announce Type: cross Abstract: Lossless acceleration schemes, such as speculative decoding, promise significant inference speedups by relying on dynamic token-level alignment between a draft and a target model.
arXiv AI
23h ago
The Model's Tell: Measuring Context-Leakage Attack Signals with Behavior Gauges
arXiv:2608. 17829v1 Announce Type: cross Abstract: LLMs increasingly rely on external contexts, such as pre-defined system prompts or retrieved documents, to improve generation quality.
arXiv AI
Jul 23
HijackKV: New Threat in Position-Independent KV Cache Reuse
arXiv:2607. 19957v1 Announce Type: cross Abstract: Key-Value (KV) cache reduces inference latency in large language models (LLMs).