Quantization, distillation, pruning and serving work aimed at the same accuracy for less memory, latency and money.
arXiv:2609.08638v1 Announce Type: cross Abstract: An action chunk can span several stages of a manipulation task, yet a label for its first step describes only the current stage. We introduce Chunk-A...
arXiv:2609.06106v1 Announce Type: new Abstract: Heterogeneous Federated Learning (HFL) aims to train models across devices with diverse resource budgets while preserving data privacy. Existing HFL me...
FrogNano is a 4B coding agent trained exclusively with reinforcement learning on about 1,500 synthetic software engineering environments. Its training leverages an online task synthesis pipeline that generates tasks at the current agent’s learnability frontier, improving performance without distilling from larger models. The report details the methodology, evaluates the agent across diverse environments, and analyzes its effectiveness as a lightweight coding agent for minimal hardware.
arXiv:2605.01192v2 Announce Type: replace Abstract: An overcomplete code packs $F$ features into $d<F$ dimensions, so linear readout of one feature picks up cross-talk from the others, bounded below...
arXiv:1907.06994v2 Announce Type: replace-cross Abstract: Mixtures of experts (MoE) are conditional mixture models in which both the mixing proportions and the component densities depend on the predi...
arXiv:2609.06893v1 Announce Type: cross Abstract: Direct preference optimization DPO is a promising offline approach for aligning large language models (LLMs) due to its simplicity, computational eff...
The paper introduces a contrastive learning approach for anomaly detection in multi-illumination and multi-focus display images. It builds on Multiresolution Knowledge Distillation (MKD) and proposes Multiresolution Contrastive Distillation (MCD), which eliminates the need for explicit positive/negative pairs by adjusting distances between teacher and student features. A blending module aggregates multi-channel data into a three‑channel input, and the method achieves superior AUROC and accuracy on the MMdAD dataset compared to state‑of‑the‑art baselines.
arXiv:2609.10456v1 Announce Type: new Abstract: Nanoscale imaging of mammalian brains is critical for connectomics. X-ray laminography enables high-throughput imaging of extended, plate-like biologic...
The paper introduces RouteOPD, a new on‑policy distillation method that treats teacher‑student disagreement as a probability transport problem. By decomposing disagreement into excess sources and deficit destinations, RouteOPD pairs them explicitly and optimizes pairwise log‑odds toward targets derived from a bounded teacher potential. Experiments on four teacher‑student configurations and four mathematical‑reasoning benchmarks show that RouteOPD consistently outperforms sampled reverse‑KL OPD, achieving higher routing fidelity and lower background leakage.
The paper addresses the challenge of long input contexts in Retrieval-Augmented Generation (RAG) systems, where concatenating many retrieved chunks increases prefill workload and time to first token (TTFT). It proposes a dual strategy: fine‑tuning the model to be aware of KV cache concatenation and selectively recomputing only part of the KV caches. Experiments on the RULER benchmark show that for a 124k‑token input, this combined method boosts the RULER score by 9.7 points over a baseline that recomputes caches only, while cutting TTFT by 80% compared with full attention.
arXiv:2609.07009v1 Announce Type: new Abstract: Multimodal continual learning has recently shown great potential for developing agents with human-like intelligence by continuously learning new tasks...
The paper introduces Stable-MM-R1, a framework that stabilizes reinforcement learning for multimodal reasoning by addressing training instability and entropy collapse. It proposes Potential‑Aware Query Mining (PAQM) to filter data toward high‑potential samples and Hybrid Stratified Replay (HSR) to restructure batches using path entropy and reward stratification, reusing stability anchors and hard negatives. The method demonstrates superior performance on complex reasoning tasks compared to strong baselines.
arXiv:2609.07956v1 Announce Type: new Abstract: Tabular Foundation Models (TFMs) have recently demonstrated strong predictive performance through in-context learning, but their deployment in high-thr...
The paper critiques current evaluations of efficiency methods for large language model–based multi‑agent systems, arguing that reported gains are often inflated by method‑specific prompts and starting topologies. It introduces a controlled, MAS‑demanding diagnostic benchmark that standardizes the backbone model, agent registry, and runtime, and systematically varies topology, scale, depth, and tool use. The authors find that many claimed efficiency improvements are setup‑dependent, sometimes stemming from structural collapse or random pruning rather than genuine, robust gains.
CALM introduces a smooth trust gating mechanism for decentralized federated learning, replacing hard filtering of teacher models with class‑wise, sample‑wise, and label‑based weighting. It allows clients with heterogeneous architectures to distill knowledge from peers without a central server or shared data, even under severe non‑IID label skew. Experiments on CIFAR‑10, SVHN, OrganAMNIST, and Google Speech Commands show that CALM consistently outperforms uniform and hard‑filtered distillation and matches or exceeds other heterogeneous‑FL methods.
arXiv:2609.07408v1 Announce Type: new Abstract: This study investigates the classification of individuals as healthy or at risk of Parkinson's disease using machine learning (ML) models, focusing on...
arXiv:2606.13945v2 Announce Type: replace Abstract: Rare diseases affect over $300$ million patients across more than $7{,}000$ conditions, yet no single hospital encounters enough cases of any one c...
arXiv:2609.06557v1 Announce Type: new Abstract: Large language models (LLMs) are often considered fragile under aggressive sparsification, and maintaining reliable performance typically requires stic...
PAC-CF introduces a method called Probably Approximately Correct Conformal Filtering to calibrate irreversible frontier pruning in large language model (LLM)-guided search. The approach treats tree pruning as a PAC‑guaranteed decision problem, deriving a conformal margin from score deficits on held‑out Native traces and applying a direct score‑gap filtering rule during deployment. Experiments across multiple domains and controllers show that PAC‑CF improves utility at various budgets and reduces workload metrics, notably boosting equal‑domain utility by 4.38 points and cutting physical requests by 18.95% with a 23.76% token reduction on a pruning‑aware ToolTree under a 100‑request budget.
The paper investigates online linear regression with sparse comparators, focusing on feature priming techniques that reweight features using past data. It establishes sparse‑regret lower bounds that invalidate sparse‑logarithmic guarantees for univariate, Pearson, and multivariate priming rules under a past‑only Moore–Penrose protocol, showing ≥Ω(min{T,√d}) clipped regret for unit‑power rules and linear regret for powered rules in high dimensions. The authors also provide tight rank upper bounds for certain priming schemes and present algebraic constructions yielding Ω(min{T,d^{1/4}}) regret for unit‑power multivariate priming, while noting that the exact multivariate frontier remains open.