Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.
Given a collection of $N$ graphs, graph-level clustering (GLC) seeks to partition the $N$ graphs into semantically meaningful groups, which serves as a fundamental task in analyzing graph-structured d...
Activation steering can modify language-model behavior without parameter updates, but behavioral efficacy alone does not determine whether an intervention is usable. Contrastive Activation Addition (C...
Designing systems that attribute a generated image to its source model has lately become a major area of AI research. Most existing datasets and systems related to image source attribution are targete...
Time series forecasting is critical for demand prediction and resource management in modern Web services, yet applying pretrained Time Series Foundation Models (TSFMs) to downstream tasks remains chal...
Scalaris is a scalable enterprise memory architecture designed to balance retrieval accuracy and context efficiency for conversational AI assistants. It separates user interactions and assistant responses into distinct long‑term and short‑term memory streams, each with specialized representations and ingestion/retrieval pipelines. The system achieves state‑of‑the‑art benchmark scores while reducing retrieved context tokens by about tenfold and improving overall response quality by 36 % in proprietary deployments.
PGuard is a native, streaming guardrail for multimodal large language models that integrates safety monitoring directly into the backbone model’s inference pipeline. It uses dynamic layer gating to aggregate intermediate representations for token‑level safety detection without altering generation behavior, offering lightweight and sequence‑aware variants to balance accuracy and efficiency. Evaluations on over 30 benchmarks show state‑of‑the‑art performance with minimal runtime overhead (0.82‑2.26%) on a 122B‑parameter backbone.
VisCodeBench is a new text‑to‑visualisation benchmark comprising 2,672 samples across educational, scientific, and general‑audience domains, designed to evaluate how well large language models can generate visualisations from natural‑language requests. The study benchmarks three visualisation systems—Codex, Claude Code, and LIDA—paired with various LLMs on seven metrics covering execution, chart content, and visual quality. Results show that while basic execution is near‑solved, no system reliably reproduces reference data, with the best data similarity at 0.81 and dropping to 0.69 on the hardest subset; system design and model choice both significantly influence performance, and latency remains a practical challenge.
IndicDBBench is a new multilingual Text-to-SQL benchmark that evaluates cross‑lingual semantic parsing across 20 databases and 237 tables in seven languages, including English and six Indic languages. The benchmark uses real‑world open‑data schemas from NDAP and IDP, and a three‑agent pipeline to create complex relational structures with high relational density and join depths up to six. Evaluation of state‑of‑the‑art models shows a 9‑point drop in execution accuracy from English to Indic languages, revealing an "Indic Gap" that persists even when gold tables and literal values are supplied.
ElectroChemLLM is a multimodal large language model pipeline that extracts structured electrochemical records—such as electrolyte compositions, electrode materials, cell configurations, and measured physicochemical properties—from unstructured scientific PDFs. By combining hybrid document parsing with vision-capable LLM orchestration, it captures data across text, tables, and figures, achieving an F‑1 score of 0.97 on a benchmark of 232 battery‑electrolyte papers. The extracted data populate ElectroChemDB, a database of 15,000 research articles, which is then used to train machine learning models for electrolyte property prediction, demonstrating a full workflow from literature to materials discovery.
The paper introduces TRIO, a retrieval‑augmented generation framework that uses three agents—query extension, judge, and answer—to iteratively refine queries and improve retrieval and answer quality. It aligns each agent’s objective with rewards derived from answer quality, employs unanimity voting to label expanded queries, and applies dynamic matching‑based rewards with curriculum learning for the answer agent. Experiments on multiple benchmarks show TRIO outperforms state‑of‑the‑art methods.
The paper introduces Student‑Prefix Teacher Adaptation (SPTA), a method that trains a teacher model to better handle student‑generated prefixes before performing on‑policy distillation (OPD). By collecting prefixes of random lengths and using reinforcement learning with verifiable rewards (RLVR), the teacher learns to recover from student‑induced states rather than simply imitate the student’s reasoning. Experiments on mathematical reasoning benchmarks show that SPTA improves student performance compared to standard OPD and OPD with task‑level RLVR‑adapted teachers.
The study investigates how different selection rules affect the performance of adaptive halting in looped language models, specifically comparing a trained stability probe with Ouro's native exit gate on a frozen Ouro-1.4B model. In a pre‑registered confirmatory experiment, the probe saved more of the four‑loop budget than the native gate, achieving a gross macro saving of +0.053 across three benchmarks, but a secondary point‑only analysis reversed this advantage. Additional comparisons showed no clear benefit over an untrained KL‑divergence baseline, and the probe performed poorly on an out‑of‑distribution benchmark, highlighting the importance of jointly evaluating halting signals and selection procedures.
The paper introduces Discrete Latent Reasoning (DLR), a method that transforms continuous latent states into explicit discrete tokens by rendering textual chains of thought into images, extracting visual features, and clustering them to build a discrete latent vocabulary. DLR enables standard autoregressive modeling over both natural language and latent tokens, supporting pretraining alignment, supervised fine-tuning, and reinforcement learning. Experiments on five reasoning benchmarks and two model series (Qwen3‑VL and LLaMA‑3) show that DLR outperforms prior latent reasoning baselines with up to 20× compression while preserving interpretable semantic structure.
The paper surveys the evolving field of agent skill evaluation and evolution, emphasizing the shift from isolated skill creation to automated, evaluation-driven development. It categorizes skill evolution into four paradigms—execution feedback, trajectory distillation, compression, and reinforcement learning—and analyzes six skill-centric benchmark categories to highlight gaps and trade-offs. The authors outline open research directions for building generalizable, efficient, and verifiably safe skill ecosystems.
The paper introduces Concise-9B, a version of the Qwen3.5-9B reasoning model that reduces unnecessary thinking tokens through concise self‑training. By applying two rounds of on‑policy rejection‑sampling fine‑tuning and a small share of answer‑synthesis targets, Concise-9B cuts token usage to roughly two‑thirds of the original model while maintaining comparable accuracy on a wide range of hard mathematics and reasoning benchmarks. The study also shows that most of the token savings occur after the answer has already formed, and that the approach’s effectiveness depends on the verbosity of the base model.
TMax is an open reinforcement‑learning recipe that trains terminal‑using agents from small language models, achieving 33% on Terminal‑Bench 2.0 with only 9B parameters and outperforming larger prior models. The approach generates training data via a novel taxonomy that blends difficulty control, personas, and verifier diversification, enabling cheap creation of many terminal environments for both RL and supervised fine‑tuning. The recipe is applicable to models ranging from 2B to 27B across different families, dominating the Pareto curve for open terminal‑agent performance at smaller sizes.
The paper investigates how structured pruning of large language models affects reasoning performance and sampling efficiency. Across expert, feed‑forward, and layer pruning on mathematical and coding benchmarks, the authors find that heavily pruned models can match or surpass the original model’s reasoning coverage, yet they often require more attempts to sample correct answers, leading to higher token usage. The study highlights a trade‑off between maintaining problem coverage and the cost of obtaining accurate solutions, suggesting that pruning should be evaluated on both dimensions.
The study compares two Norwegian language models—Borealis 2, further‑trained from Gemma 4, and Gemma 4 itself—on how often they provide outdated numeric values from Norwegian statutes and regulations. Using the closed norregel benchmark, the models were evaluated on 503 primary items; Borealis 2 was outdated on 10 items and Gemma 4 on 11, a difference that is not statistically significant and falls within the pre‑registered smallest effect of interest. The only secondary measure that differed after Holm correction was the share of answers giving a value that took effect after the question date, with Gemma 4 giving such answers more often.
The paper introduces three hierarchical frameworks for community detection in complex networks, integrating linkage-based clustering, vertex proximity matrices, and graph representation learning. It evaluates 364 module combinations on synthetic and real datasets, finding that pipelines using Wasserman‑Faust, PPMI proximity, and DNGR representations match the performance of Leiden and Louvain while producing nested partitions. The approach excels on graphs with many small communities but is limited by its time complexity to networks of roughly 10,000 vertices on personal machines.
MEMENTO is a long‑context agent built on Qwen3‑8B that learns to manage its fixed‑size context window through compaction, replacing its history with self‑generated notes. The model is first fine‑tuned on synthetic trajectories with hindsight notes and then reinforced with a reward combining answer correctness, note quality, and confidence calibration. At test time, MEMENTO can also fold context and backtrack based on confidence, achieving a 67.8% accuracy on book‑level QA (up to 2.5 M tokens) while never holding more than 20 K tokens, outperforming GPT‑5.5 and GLM‑5.3 agents at significantly lower cost.