Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.
The paper proposes ten desiderata for designing and evaluating reinforcement‑learning agents that learn continuously over a lifetime. It reviews existing approaches against these criteria, noting that no single agent has yet integrated all ten. Additionally, it introduces a compact persistent‑world scenario and a benchmark to test multiple behavioral capabilities within one continuous lifetime.
The paper demonstrates that the temporal speckle of broadband light can encode probabilistic weights for Bayesian neural networks, allowing a photonic processor to perform probabilistic convolutions every 37.5 ps. By programming mean and variance into independently amplitude‑bandwidth‑controlled spectral channels, the system delivers uncorrelated samples without heavy random‑number generation. The resulting ultrafast photonic AI classifies blood‑cell images while detecting out‑of‑distribution samples and distinguishing epistemic novelty from aleatoric ambiguity on standard uncertainty benchmarks.
The paper introduces Trainable Vector Quantization (TVQ), a method that uses straight-through estimation to allow gradient descent optimization of vector-quantized large language models. TVQ reformulates VQ with trainable latent weights, enabling arbitrary objective minimization and is applied in a block-wise knowledge distillation framework to balance quality and training efficiency. Experiments show TVQ achieves state‑of‑the‑art low‑bit post‑training quantization, achieving near‑lossless 2‑bit quantization for LLaMA2‑70B with only a 1% accuracy drop on question‑answering benchmarks.
PrivacyFlow introduces a new benchmark, PrivacyBench, and a defense system that separates privacy planning from task execution in autonomous LLM agents. The system traces data provenance, models recipient knowledge, and simulates inference attacks to weigh disclosure risk against task utility, allowing it to minimize or block data sharing. Experiments show that PrivacyFlow reduces privacy failures and increases completed task sequences by 22–55 percentage points, while revealing a tradeoff between privacy, utility, and user autonomy.
The paper introduces $ abla$Mimic, a method that uses exact gradients from a differentiable simulator to train humanoid control policies more efficiently. It mitigates gradient explosions by clipping per‑sample gradients and reuses simulated windows via multistep regression, while the NablaEngine GPU simulation engine computes full contact dynamics and adjoints in a single megakernel, achieving an 11.5× speedup over MuJoCo Warp. On the Unitree G1, $ abla$Mimic matches state‑of‑the‑art world‑model performance in 20 minutes and surpasses it in 1.8 hours, outperforming standard first‑order baselines that fail due to exploding gradients.
The paper introduces ORACLE, an online reasoning framework designed to anticipate smartphone scams early by analyzing partial, streaming app‑usage trajectories. It presents a real‑world benchmark of 12 scam types across 95 apps and 15‑day periods, and employs a self‑evolving context manager plus an on‑policy self‑distillation scheme to consolidate fragmented evidence and enhance early detection. Experiments demonstrate that ORACLE consistently improves early scam anticipation, providing timely warnings while reducing false alerts in realistic streaming scenarios.
The paper presents a controlled benchmark of nine deep‑learning architectures for multi‑horizon financial forecasting, using 918 runs across 12 assets and two forecast horizons. It employs a five‑stage evaluation pipeline that isolates architectural effects from hyperparameter tuning, random seeds, and asset characteristics. Results show ModernTCN and PatchTST consistently outperform others, with ModernTCN achieving the lowest RMSE in 75% of asset‑horizon points and a statistically significant global ranking advantage.
The paper investigates how tool-call traces from large language model agents can reveal user attributes. Using the PrivacyTrace benchmark, the authors show that an attacker can infer user profiles with a 26.25% success rate from tool-call data, outperforming prior guessing methods. The study highlights that tool arguments and results are key evidence, and linking traces across sessions increases profiling risk.
The paper proposes a systematic framework for one‑shot pruning of experts in Mixture‑of‑Experts language models, focusing on routing frequency, gate weighting, and activation strength. It derives a selection principle that recommends task‑agnostic pruning use activation‑based criteria without gate information, while task‑specific pruning can incorporate routing and gate data. Two new task‑agnostic criteria, Mean Activation Norm (MAN) and Mean Squared Activation Norm (MSAN), are introduced and shown to outperform existing methods across multiple MoE models and benchmarks, improving average performance by up to 8.8 points.
The paper introduces LIDFS, an unsupervised feature selection technique that uses Local Intrinsic Dimensionality (LID) to rank features based on their local discriminability. By treating each feature as a point and estimating its LID via maximum likelihood from distances to its k nearest neighboring features, LIDFS assigns higher importance to features with lower LID and refines the ranking to remove locally redundant features. Experiments on 23 high‑dimensional datasets show that LIDFS outperforms both classic and state‑of‑the‑art methods in supervised and clustering evaluations while running faster than its closest competitor.
RetryRoute is a new routing framework for web agents that edit images by calling open editing models in a critique-and-retry loop. It addresses the problem that existing routers treat each retry as an independent draw, whereas empirical analysis shows that a retry after a failure succeeds only 17‑53% as often as a first try across six open models. By estimating each model’s chance of success given the request’s failure history, planning retries, escalations and abandonment, and incorporating live queue waits, RetryRoute improves success rates and delivery within service-level objectives compared to eleven prior baselines.
The paper evaluates 14 state‑of‑the‑art text‑to‑video retrieval models on three major datasets using a unified framework. It finds that short, clear, and simple captions—especially those describing single actions or color attributes—yield higher recall, while complex, multi‑step events or fine‑grained scene descriptions remain difficult for all models. Attention‑driven architectures better handle temporally dependent or multi‑step queries, whereas dual‑encoder and multimodal fusion models excel mainly on simpler captions, and cross‑dataset generalization improves with larger, more diverse caption sets but generative captions do not consistently boost accuracy.
The paper introduces a new multi‑agent reinforcement learning framework that separates agent identity from behavior by using events to trigger behavioral changes. It proposes Neural Manifold Diversity (NMD) to create a continuous behavior manifold and an event‑based hypernetwork that generates Low‑Rank Adaptation (LoRA) modules for on‑the‑fly policy reconfiguration. The authors prove that this design preserves reward maximization and show empirically that it outperforms baselines, generalizes zero‑shot, and uniquely solves tasks requiring sequential behavior reassignment.
arXiv:2608. 05595v2 Announce Type: replace-cross Abstract: Circuit cutting lets a large quantum neural network (QNN) run as independent subcircuits on small devices, but rebuilding its outputs by quasiprobability reconstruction carries a classical sampling overhead exponential in the number of cuts.
The paper presents a two‑stage approach to street‑panorama semantic segmentation using Vision Transformer ensembles. In the first stage, nine pretrained models are compared under equal compute budgets, and the best two are trained with three random seeds, achieving 60.95% mIoU on validation and 57.08% on the hidden test leaderboard. The second stage explores augmentations, larger inputs, and additional architectures, culminating in an ensemble of four models that scores 64.53% mIoU on validation and 61.54% mIoU (71.93% F1) on the hidden test leaderboard, while the study documents all runs, costs, and releases the code and models.
Qistas is a new protocol designed to evaluate how large language models treat the social, confessional, and regional constituencies of a society in a portable, pre‑registered manner. The pilot implementation focuses on Iraq and measures procedural symmetry, differential refusal, stereotype reliance, and terminological framing using paired statistical tests and a power analysis that specifies detectable effect sizes before data collection. The authors release an authoring apparatus—including a taxonomy, checklists, audit logs, and a human review notebook—and report a field test that uncovered several defect patterns common to counterfactual fairness benchmarks.
The paper investigates how conversational Retrieval-Augmented Generation (RAG) benchmarks that rely on relevance judgments to determine context sufficiency can be misleading. By re‑running official baselines on the RETECO track and evaluating with both large‑language‑model judges and a blind audit, the authors show that stronger retrievers produce a higher proportion of contexts lacking gold documents yet still containing the answer, leading to a sharp drop in shortcut precision from 0.68 to 0.10. The study demonstrates that many false misses arise from hard negatives, gold content from other turns, or neighboring chunks, and that using retrieval scores alone as a sufficiency detector can underestimate answerability by up to 0.19 in AUROC.
HyperRepair is a framework that restores the general capabilities of fine‑tuned language models while keeping their domain specialization, using only inference time. It learns a hypernetwork from repairs on source models, which generates a Repairing Side Net (RSN) that selectively corrects the model based on activation fingerprints. Evaluated on two model families, 12 LoRA adaptation datasets, and 15 general benchmarks, HyperRepair retains 99.5% of fine‑tuned domain performance and 93.4% of the base model’s general performance.
The paper introduces Grounded Generative Speech Recognition (GSR), redefining speech recognition as generating semantically grounded text rather than a single transcript. It defines a grounded reformulation space and a four-level hierarchy of reformulation capabilities, and presents GSR-Bench, a comprehensive benchmark covering diverse scenarios such as meeting transcription, cross-dialect translation, and robustness to silence or noise. The authors also develop a multi-dimensional evaluation framework that combines speech grounding assessment with LLM-based semantic judgment, validated by human preference studies, and analyze trade-offs and failure modes across various ASR architectures.
The paper introduces a unified training objective that blends focal loss with supervised contrastive learning to improve automatic CEFR level assessment. This approach addresses class imbalance and ambiguous level boundaries by emphasizing hard examples and learning discriminative representations. Experiments on sentence- and document-level CEFR assessment in both monolingual and multilingual contexts show that the method outperforms strong baselines and sets new state‑of‑the‑art results on multiple benchmarks.