Tool use, function calling, orchestration and the protocols that let models act rather than only answer.
Formal proofs in Lean 4 that pass the kernel's type checker can nonetheless vary widely in quality. We introduce ProofJudge, an agentic LLM-as-judge system that scores formal proof quality along five...
Research: smolmachines / smolvm as a sandbox for untrusted Python & JavaScript I tasked Claude Fable 5 running in Claude Code for web with the following research task: Put https://smolmachines. com through its paces as a fast secure sandbox.
Last week I recorded an episode of the Talking Postgres podcast with Claire Giordano on the subject of "How AI is changing software development". We had a really great conversation.
SCAPE is a scenario‑conditioned simulation‑augmented policy evaluation framework that predicts real‑world policy performance for specific scenarios using limited paired simulation‑and‑real samples and extensive simulation rollouts. It corrects sim‑to‑real bias in simulation labels before training the prediction model and calibrates prediction uncertainty via conformal prediction. Experiments on autonomous driving and quadruped velocity tracking show SCAPE reduces scenario‑level prediction error, improves testing sample efficiency, narrows calibrated prediction intervals, and generalizes better to out‑of‑distribution scenarios, enabling fine‑grained deployment strategies.
The paper evaluates browser-based large language models (LLMs) for extracting detailed, contextualized data from scientific papers. It presents four workflows: (1) expert-curated prompts yield good extraction but struggle with nuance; (2) LLMs can generate effective prompts from simple instructions; (3) autonomous literature discovery is challenging, with missing or hallucinated references; (4) LLMs can build new datasets from guidelines that align closely with human experts, yet still need human oversight. The study outlines a practical, auditable workflow where experts set standards, models cross-check extractions, and researchers resolve disputes, enabling scalable scientific data curation.
The paper introduces Harness Continual Learning (HCL), a paradigm where an agent’s state evolves through prompts, memories, tools, skills, and routing rules while keeping the foundation model frozen. HCL defines harness-level forgetting and proposes guarded harness evolution with a Continual Optimizer and Evaluator to balance improvement, retention, and validity. Experiments across textual reasoning, multimodal perception, and open‑world interaction show over 10% performance gains and demonstrate explicit control over the stability–plasticity trade‑off.
SkillForge is a self‑distillation framework that proactively builds project‑specific knowledge for large language model agents by synthesizing and resolving artificial issues derived from a repository’s test‑covered core functionalities. Rather than waiting for real issues to reveal knowledge gaps, SkillForge generates these synthetic problems, learns reusable entity‑grounded skills, and associates them with relevant repository entities. Experiments with both open‑source and closed‑source models show that this proactive knowledge acquisition consistently outperforms strong baselines in issue resolution tasks.
TestifAI is a deep learning testing framework that estimates model robustness against combinations of input perturbations efficiently. It allows users to define structured spaces of semantic perturbations and severity levels, then query robustness for any combination. By employing partial model tomography, TestifAI reconstructs higher‑order perturbation effects from low‑order tests, achieving less than 7% error while cutting inference counts by 60‑80% on image and language classification tasks.
TestifAI is a deep learning testing framework that estimates model robustness against combinations of input perturbations efficiently. It allows users to define structured spaces of semantic perturbations and severity levels, then query robustness for any combination. By using partial model tomography, TestifAI reconstructs higher-order perturbation effects from low-order tests, achieving less than 7% error while reducing inferences by 60–80%.
Last-mile delivery aims to handle dynamically arriving orders with couriers while modeling complex spatial and temporal correlations. Recent learning-based methods model spatiotemporal dependencies am...
The paper introduces RoomWright, a code‑driven framework that generates 3D indoor scenes for embodied AI by focusing on functional usage rather than just visual layout. It performs usage‑driven object reasoning, treating anchors as task centers to select task‑required objects and their affordances, and compiles interactions into trigger‑condition‑effect rules that update object states. The system also addresses ambiguous object orientation through annotation‑guided usage cues, producing scenes that are executable, editable, and ready for simulation‑based policy learning.
MemFuse introduces a multi‑source memory fusion system and a corresponding benchmark, MemFuseBench, designed to evaluate agents that must integrate fragmented observations from multiple applications, devices, users, and time points. The benchmark uses a Scene‑to‑Sensor pipeline to generate source‑tagged observations, evidence‑grounded questions, and adversarial distractors, enabling systematic assessment of temporal reasoning, cross‑source evidence fusion, and noise robustness. Experiments show that MemFuse outperforms other memory systems across all evaluated LLM settings, especially on questions requiring cross‑source evidence fusion.
DynCur-Geo introduces a dynamic curiosity framework for active geo‑localization, adjusting the intrinsic reward based on the remaining distance to a target. A distance‑aware gate promotes early exploration and transitions the policy toward goal‑directed behavior as the UAV approaches the target, while potential‑based reward shaping provides dense progress guidance. Experiments in multimodal, cross‑scene, disaster‑affected, and long‑range scenarios demonstrate consistent performance gains over existing active geo‑localization baselines.
DART‑SD introduces a diamond‑topology aware framework for training large language models to perform multi‑turn tool‑calling. It models the execution process as an Interaction‑State Transition Graph, identifies critical topological breakpoints, and retrieves recovery references to guide localized self‑distillation. Experiments show that this approach outperforms traditional full‑trajectory baselines on complex multi‑turn tool‑calling benchmarks.
GraphWake demonstrates a new threat called Memory‑Mediated Polarization Cascade, where LLM‑driven agents use their memory to persistently store and later reproduce arguments that reinforce their stances. The attack unfolds in three stages: exposure and memory retention, retrieval and reproduction during neutral discussion, and iterative propagation to untreated agents. Experiments show that GraphWake significantly increases group polarization across various discussions and memory systems.
CityReal is a modular framework that uses large language model agents to simulate human-aligned urban behavior. It models agents as intention-driven decision makers who pursue coherent mobility and activity plans, learning habits and preferences over time. By training textual adapters to align agent decisions with observed population statistics, CityReal improves realism at both micro and macro levels and can scale to tens of thousands of agents for analyzing crowd density, place popularity, mobility flows, and well‑being under various urban scenarios.
The paper introduces Porting Benchmark, a curated dataset of 1,234 security patch backporting cases that span cross-version, cross-branch, and cross-repository scenarios, along with a common evaluation framework. Five tools—spanning program analysis, LLM prompting, and LLM agents—are evaluated under aligned settings, revealing that performance varies significantly across tools and that complex patches (Type-IV) see a sharp drop in success rate. The study identifies four root-cause categories for failures and demonstrates that reference-based benchmark scores may not fully capture real-world remediation, as executable validation uncovers additional integration issues.
The paper introduces GADR, a multi‑agent, self‑correcting workflow that extracts architectural decisions from raw meeting transcriptions and produces Nygard‑formatted ADR drafts. A feasibility study using five real project transcripts, expert reviews by four senior architects, and evaluations by fifteen students shows that GADR captures most expert‑identified decisions and yields drafts that participants find clear and useful, outperforming zero‑shot and few‑shot baselines in stability and structural adherence. The study also highlights a trade‑off: RAG‑based enrichment can deepen ADR content but may introduce transcript‑unfaithful material, raising open questions about traceability in automated architectural documentation.
MobileWorldSafety is a benchmark that evaluates the safety of large language model–powered GUI agents on Android by exposing them to 142 real-world risk tasks involving environmental injection attacks. The benchmark uses a two‑stage verification pipeline—rule‑based checks for clear cases and an LLM judge for ambiguous ones—to distinguish safety failures from capability failures. Experiments on six agents show high vulnerability, with attack success rates between 40.4% and 66.9%, highlighting that current agents often fail to remain safe when faced with adversarial content presented as normal mobile context.
The paper introduces policy‑counted DecPOMDPs, a variant of decentralized partially observable Markov decision processes that mitigates the exponential growth in agent numbers by counting policies instead of agents. By exploiting symmetry among agents, the authors achieve a compact representation that reduces model complexity and evaluation cost to polynomial levels. They further present a dynamic programming algorithm that leverages this compact form to solve policy‑counted DecPOMDPs efficiently.