arXiv AI
SerenAI: State-transition system inspired by text-based world AI models
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Jul 28
Confidently Wrong: Exception Chain Collapse in Frontier LLM Rule Evaluation
arXiv:2607. 23386v1 Announce Type: new Abstract: We document a failure class in frontier large language models -- exception chain collapse -- observed in eligibility evaluation under nested conditional rules of the form "A is required UNLESS B applies, UNLESS C overrides B".
arXiv AI
Aug 24
No Judgment Without a Reason: Counterfactual Receipts for Versioned AI Evaluators
The paper introduces a framework for evaluating AI systems that not only checks final labels but also tracks the reasoning behind them through three core sources—grounds, norms, and authority—forming an eight-cell counterfactual judgment cube. It defines minimal source replacement sets, called judgment receipts, to explain changes in verdicts and provides certification cost bounds for black-box evaluators. The authors present ReasonBench, a benchmark with 19,520 cases, and demonstrate that while high standard accuracy can mask robustness issues, receipt accuracy reveals significant gaps in reasoning consistency across different models.
arXiv AI
Sep 2
SAGE: State-Grounded, Abstention-Aware Evaluation of Task-Oriented Dialogue Agents
SAGE (State‑Grounded, Abstention‑Aware Evaluation) is a new framework for assessing task‑oriented dialogue agents that focuses on whether each turn correctly advances the workflow state rather than just the quality of the reply. It compiles workflow specifications and per‑turn state differences into schema‑grounded criteria, then evaluates them through a cascade of symbolic rules and encoder/NLI verifiers that abstain instead of guessing, producing a turn‑level decision with an evidence trace. In experiments across MultiWOZ, Schema‑Guided Dialogue, and ABCD datasets, SAGE‑Core—using only symbolic rules and on‑device encoders—outperforms all evaluated LLM‑based judges, including GPT‑4.1 variants, while incurring zero paid LLM cost.
arXiv Machine Learning
Sep 2
CRAFT: Fine-Tuning Pre-hoc Explainability in AI-native 6G RAN
The paper introduces CRAFT, a data‑centric fine‑tuning approach that aligns small language models (SLMs) for pre‑hoc reasoning in AI‑native 6G radio access networks (RAN). By automatically generating verified (input, trace, label) triplets and fine‑tuning with low‑rank adaptation, CRAFT achieves high accuracy and F1 scores on TRACTOR and IC xApp datasets while avoiding parse failures that plague RL methods like GRPO. It also reduces energy consumption by 59% compared to GRPO baselines, offering a more sustainable path to auditable AI in 6G RAN.
arXiv AI
2d ago
Detecting Inconsistencies in Model Specifications with LLM-as-Verifier Reasoning
arXiv:2610.01847v1 Announce Type: cross Abstract: Model specifications define how large language models (LLMs) should behave, guiding alignment training, inference-time behavior, and evaluation. Yet...
arXiv AI
Sep 24
Verifiable Hidden Dynamics Play: Generating Agentic RL Environments from Solved Mechanisms
The paper introduces VHD-Play, a pipeline that first samples and solves a mathematical model before generating agentic reinforcement learning environments, ensuring that dynamics and evaluation are aligned from the outset. This approach yields 3,300 diverse environments at a low cost and significantly improves the performance of a large language‑model agent (Qwen3.6‑35B‑A3B) across multiple diagnostic families and external benchmarks. The study demonstrates that stateful interaction is a key factor in learning gains and that scaling the training substrate can further enhance performance.