arXiv AI
Aug 18
pico-type: A 1.5M-Parameter Byte-Level Multi-Head Content Classifier
arXiv:2608. 14658v1 Announce Type: cross Abstract: We introduce pico-type, a byte-level multi-head content classifier with approximately 1.
MimeLens: Position-Agnostic Content-Type Detection for Binary Fragments
arXiv:2606. 04171v1 Announce Type: cross Abstract: File-type classification underlies many workflows like malware triage, forensic carving, packet inspection, and storage indexing.
Related stories
arXiv Computation and Language
Sep 16
Where Post-Training Quantization Breaks Text Embedders: A Measured Map Across Four Embedder Families
arXiv:2609. 16391v1 Announce Type: cross Abstract: Weight-only post-training quantization is the cheapest way to shrink a retrieval embedder, and the received advice for applying it -- protect the embedding table, allocate bits by module sensitivity, prefer a ranking-aware objective over weight reconstruction -- was carried into LLM quantization largely intact.
arXiv AI
Aug 21
Daedalus-150M: A Convolution-Attention Hybrid Designed for CPU Inference
arXiv:2608. 20210v1 Announce Type: cross Abstract: Small language models are usually built like large ones and then squeezed onto a CPU afterwards.
arXiv Machine Learning
Sep 14
Breaking the Token Ceiling: Distilling Smaller, Stronger Byte Models
The paper investigates whether small models distilled from larger ones behave similarly when using byte versus token tokenization. It introduces two methods—Marginalize‑It (approximate) and End‑Of‑Token (exact)—to convert token logits to byte logits, and conducts a large‑scale study on decoder‑only dense transformers ranging from 1 billion to 1 trillion bytes of data. Results show that while token‑based models excel early, byte‑based models eventually surpass them with more compute, achieving higher performance ceilings, greater data efficiency, and lower logit storage costs.
arXiv AI
Jul 14
What Context Does a Coding Agent Actually Need to Act?
arXiv:2607. 09691v1 Announce Type: cross Abstract: A modern coding agent can hold an entire repository in its context window.
Hugging Face Trending Papers
Aug 20
Daedalus-150M: A Convolution-Attention Hybrid Designed for CPU Inference
Small language models are usually built like large ones and then squeezed onto a CPU afterwards. We did the opposite: we fixed the target first, one user, one token at a time, 4-bit weights, ordinary CPU, and chose the architecture to suit it.
arXiv Machine Learning
Jul 28
Cross-Attention Calibrated Deduplication for Retrieval-Augmented Generation System
arXiv:2607. 24332v1 Announce Type: cross Abstract: Common chunking strategies in Retrieval-Augmented Generation (RAG) systems often create redundant chunks.
arXiv Computation and Language
Sep 22
An Empirical Cost Attribution of Context-Compression Gateways in Multi-Turn Coding Agents
arXiv:2609.22114v1 Announce Type: new Abstract: Context compression is widely proposed as a way to cut the token bill of LLM coding agents, and public benchmarks report that aggressive compression pr...
arXiv Machine Learning
Aug 4
Leak It: A Probabilistic Approach to Training-Data Extraction from Black-Box Language Models
arXiv:2608. 00144v1 Announce Type: new Abstract: Membership inference (MIA) on language models is usually summarised by an aggregate ROC-AUC, but such evaluations are confounded: model-free blind baselines separate members from non-members from surface text alone.
arXiv Machine Learning
Jul 8
Multi-Channel Spread-Spectrum Code Watermarking
arXiv:2607. 06009v1 Announce Type: cross Abstract: Attributing code to the large language model that produced it is essential for provenance, licensing, and misuse accountability, yet no deployed watermark meets this need.
arXiv Machine Learning
Jul 7
Walma: Learning to See Memory Corruption in WebAssembly
arXiv:2603. 24167v2 Announce Type: replace-cross Abstract: WebAssembly's (Wasm) monolithic linear memory turns a single memory-corruption bug into a bidirectional threat: a compromised module can attack its embedding host, and a malicious host can tamper with a trusted module's state.
arXiv Machine Learning
Jul 9
Fractal KV-Cache Archives: Lossless Symbolic Storage with In-Place Retrieval for Long-Context LLM Inference
arXiv:2607. 07144v1 Announce Type: new Abstract: The key-value (KV) cache dominates the memory cost of long-context autoregressive inference, and a growing body of work compresses it through quantization, eviction, or offloading.