arXiv Machine Learning
Jul 27
Parameter-free Adaptive Sparse Attention via Compression-Based Content Selection
arXiv:2607. 21752v1 Announce Type: new Abstract: Data-adaptive sparse attention masks substantially outperform fixed patterns (e.
Large language models
Model releases, architecture work and prompting research on large language models — from frontier-lab announcements to the arXiv papers behind them.
arXiv Machine Learning
Jul 27
RED-PIM: Reducing Data Movement for Transformers using Processing-in-Memory
arXiv:2607. 21731v1 Announce Type: new Abstract: Transformers are widely used across many domains, including natural language processing, computer vision, web search, and DNA sequence analysis.
arXiv Machine Learning
Jul 27
LeAct: Learning to Reason from Expert Actions
arXiv:2607. 21856v1 Announce Type: new Abstract: Modern reasoning models depend on reasoning data, today sourced from human annotations or distilled from stronger LLMs.
arXiv Machine Learning
Jul 27
RIS-Kernel: A Model-Agnostic Architecture for Long-Context LLM Inference via Sparse Attention
arXiv:2607. 21927v1 Announce Type: new Abstract: Full self-attention in large language models scales as O(N^2), which limits long-context document analysis to 65,536 tokens and requires costly GPU clusters.
arXiv Machine Learning
Jul 27
Teaching LLMs to Self-Evolve: Cultivating Core Meta-Skills with Reinforcement Learning
arXiv:2607. 21971v1 Announce Type: new Abstract: Test-time scaling through iterative self-evolution with environment feedback, as demonstrated by AlphaEvolve, shows remarkable performance gains.
arXiv Machine Learning
Jul 27
DCS: A Unified Conditional Sensitivity Framework for Cross-Modal Copyright Infringement Detection
arXiv:2607. 22035v1 Announce Type: new Abstract: Currently, most foundation models can reproduce or strongly depend on copyrighted training content, but output similarity alone is insufficient for infringement detection, because similar outputs may also arise from public-domain concepts, common stylistic conventions, or ordinary statistical generalization.
arXiv Machine Learning
Jul 27
Evolution-Aware MSA Reasoning for Subsampling via Factor Graphs
arXiv:2607. 22314v1 Announce Type: new Abstract: Multiple Sequence Alignments (MSAs) provide protein language models with explicit evolutionary context, but their large depth makes subsampling unavoidable under limited token budgets.
arXiv Machine Learning
Jul 27
Cross-Tokenizer On-Policy Distillation via Byte-Prefix Marginalization
arXiv:2607. 22334v1 Announce Type: new Abstract: Open-weight language models from different families exhibit complementary capabilities, motivating their consolidation into a compact student through on-policy distillation (OPD).
arXiv Machine Learning
Jul 27
IQ-JEPA: A Joint-Embedding Predictive Architecture with a Hermitian Vision Transformer for Sound Speed and Attenuation Estimation from Ultrasound IQ Data
arXiv:2607. 22351v1 Announce Type: new Abstract: The speed of sound in tissue is a prerequisite for well-focused imaging and has diagnostic value, but recovering it from raw pulse-echo channel data is fundamentally a nonlinear inverse problem.
arXiv Machine Learning
Jul 27
Interior interpretability with attention rollout: contraction and propagation profiles in Transformers
arXiv:2607. 22367v1 Announce Type: new Abstract: Feature-attribution methods assign scores relating input variables to a model's output, but do not by themselves characterize how explicitly defined interaction operators compose across its intermediate layers.
arXiv Machine Learning
Jul 27
An Explainable FFT-Based Spatial-Frequency Fusion Framework for Deepfake Detection
arXiv:2607. 17441v1 Announce Type: cross Abstract: Deepfake generation has raised growing concerns regarding digital media authenticity, misinformation, identity fraud, and public trust.
arXiv Machine Learning
Jul 27
SCOPE and SCION: A Benchmark and an Auditable Reference Pipeline for Schema Induction and Fusion from Text
arXiv:2607. 21610v1 Announce Type: cross Abstract: Schema graphs are an upstream bottleneck of schema-grounded information extraction and knowledge graph construction, yet most extraction systems assume the schema is already available.
arXiv Machine Learning
Jul 27
Procedural Knowledge Is Not Low-Rank: Why LoRA Fails to Internalize Multi-Step Procedures
arXiv:2607. 21612v1 Announce Type: cross Abstract: Parameter-efficient fine-tuning methods like LoRA have become the default for adapting large language models, succeeding across instruction following, style transfer, and factual adaptation.
arXiv Machine Learning
Jul 27
FBLayout: Optimizing Memory Layout for Efficient LLM Finetuning on Mobile GPUs
arXiv:2607. 21624v1 Announce Type: cross Abstract: Transformer-based models have enabled unprecedented capabilities across language, vision, and multimodal tasks.
arXiv Machine Learning
Jul 27
Do Modules Stay in Their Lane? Role Drift in Compound LLM Systems
arXiv:2607. 21627v1 Announce Type: cross Abstract: End-to-end reinforcement learning can improve the accuracy of compound LLM systems, but it does not constrain how modules divide labor internally.
arXiv Machine Learning
Jul 27
Small Vision-Language Models Know When They Are Wrong But Cannot Say So: A Two-Model Study of Stated versus Internal Confidence Under Realistic Image Degradation
arXiv:2607. 22034v1 Announce Type: cross Abstract: Vision-language models (VLMs) are increasingly deployed on consumer hardware where input images are degraded by compression, camera shake, and poor lighting.
arXiv Machine Learning
Jul 27
TRACE-ROUTER: Task-Consistent and Adaptive Online Routing for Agentic AI
arXiv:2607. 22465v1 Announce Type: cross Abstract: Routing to select large language models (LLMs) with different cost-quality trade-offs has become a fundamental deployment feature of enterprise AI.
arXiv Machine Learning
Jul 27
HiKV: Hierarchical Importance-Aware KV Cache with Hardware Acceleration for LLM Decoding
arXiv:2607. 22389v1 Announce Type: cross Abstract: With the rapid adoption of long-context large language models (LLMs), the continuously growing KV cache during decoding has become the critical memory bottleneck.
arXiv Machine Learning
Jul 27
LLAMA LIMA: A Living Meta-Analysis on the Effects of Generative AI on Learning Mathematics
arXiv:2601. 18685v4 Announce Type: replace-cross Abstract: The capabilities of generative AI in mathematics education are rapidly evolving, posing significant challenges for research to keep pace.
arXiv Machine Learning
Jul 27
Quantifying Political Partisanship for Cross-Platform Analyses
arXiv:2607. 21842v1 Announce Type: cross Abstract: Research on political polarization on social media depends on the ability to reliably measure partisanship in user-generated content.