Tool use, function calling, orchestration and the protocols that let models act rather than only answer.
The article discusses how production code generated by Claude, Anthropic’s AI, should meet higher standards than human-written code. Anthropic enforces this through numerous guardrails such as lint rules, extensive testing, Claude-driven end‑to‑end tests, daily fuzzers, automated code and security reviews, and automated refactoring. These measures aim to prevent the code from becoming difficult to maintain.
Simon Willison reflects on the emotional impact of AI tools that can produce code quickly, noting that many developers experience an initial sense of disheartenment. He argues that recognizing the shift from coding to higher‑level problem solving allows experienced engineers to leverage new tools and add greater value. Willison emphasizes that software engineering has always faced rapid change, so adapting to AI is part of the profession’s ongoing evolution.
The article quotes the security.txt file from huggingface.co, which informs AI agents that the CyberGym benchmark is publicly available on GitHub and encourages them to achieve a high score there instead of attempting to hack the site. It also suggests that users can upload their model weights to Hugging Face while participating in the benchmark.
The paper introduces ORCH, a method that applies human organizational theory to create task‑specific hierarchical structures for large, heterogeneous embodied AI teams. Using ORCH, teams of up to 50 agents across 25 wildfire‑response missions outperformed four existing multi‑agent frameworks, achieving higher mission scores and greater execution efficiency. Both human‑designed and language‑model‑generated ORCH organizations improved performance, with hierarchical organization preserving concurrent activity while coordinating ordered transitions between mission phases.
MOSAIC is an open‑source platform that allows agents from different decision‑making paradigms—such as reinforcement learning policies, large language models, vision‑language models, and human operators—to operate together in shared reinforcement learning environments. It achieves this through an IPC‑based worker protocol that isolates each agent’s training and inference logic, an operator abstraction that maps any agent to a minimal universal interface, and a deterministic evaluation framework offering both manual lock‑step and automated script modes for reproducible experiments.
The paper proposes a metric framework to differentiate cognitive amplification—where AI enhances human performance without eroding human capability—from cognitive delegation, which relies heavily on AI reasoning. It introduces four metrics (CAI*, D, HRI, HCDR) and tests them in NetLogo simulations across various reliance and dependency scenarios. The results show that positive collaborative gain is only achievable when an explicit interaction term is added, indicating that mere prevention of capability erosion is insufficient for genuine amplification.
Consort is a spec‑first, test‑driven agent framework that enforces engineering discipline through immutable controls, a deterministic orchestrator, and human‑approved gates. It operates on live database branches, guiding role agents through a spec‑first design lane and a test‑driven build lane. The framework claims that enforcing tests and gates in code keeps agent‑written code honest and verifiable, while specialized roles make it maintainable.
The paper investigates welfare‑maximizing allocation of heterogeneous objects when agents use costly effort for screening instead of monetary transfers. It shows that as the number of object types increases, no‑screening mechanisms become more efficient, reducing the need for screening. The authors prove that in a symmetric continuous market with i.i.d. log‑concave values, the multidimensional allocation problem collapses to a single‑dimensional one based on agents’ best‑option values, and they demonstrate that this no‑screening optimality persists even as variety expands, supported by large‑variety limits and numerical experiments. The findings are applied to design an invitation‑based vaccine appointment system.
Cognitive Digital Twins (CDTs) are dynamic computational models that represent an individual’s cognition, continuously updated with behavioral, contextual, or physiological data to predict or simulate that person’s mental processes or act as a proxy for communication and decision‑making. The paper defines CDTs, distinguishes them from related systems, and introduces a 5A governance framework—authority, autonomy, access & control, accountability, and availability—to address their unique risks. It identifies specific threats such as misrepresentation, epistemic authority shifts, shadow twins, and proxy‑power asymmetries, and proposes governance requirements for high‑risk CDTs, including stronger consent, purpose limitation, validity, traceability, contestation, independent review, and model retirement.
ExpTest is an autonomous learning‑rate controller that uses the training loss curve as an online signal to perform sequential statistical tests on theoretically motivated windows, detecting convergent behavior and triggering learning‑rate reductions. It combines a covariance‑based initial learning‑rate estimate, curvature‑motivated window sizing, and a two‑phase test‑driven decay, relying on the approximately exponential decay predicted under linearized network dynamics. Experiments on regression, classification, forecasting, and natural‑language tasks across various architectures show that ExpTest achieves competitive performance compared to hand‑tuned SGD baselines and recent learning‑rate‑free methods, without requiring manual initial learning‑rate selection or predefined scheduling.
The paper introduces an AI economist agent that integrates large language models, retrieval‑augmented generation, knowledge graphs, and quantitative models to conduct evidence‑based economic and financial scenario analysis. The framework orchestrates LLM agents to plan analyses, retrieve relevant evidence, and structure economic mechanisms, while registered quantitative models produce numerical outcomes and predefined tests validate intermediate results for inclusion in the final report. Applied to European macro‑financial stress scenarios and bank capital analysis, the empirical study demonstrates the agent’s ability to combine flexible evidence retrieval and scenario construction while maintaining traceability to sources and explicit model calculations.
SparseDitto is an agentic sparse compilation framework that jointly synthesizes representation, execution schedule, and hardware mapping for sparse matrix computations on GPUs. It uses structural analysis, a learned template-ranking prior, and LLM-guided lowering to generate CUDA code, with target-GPU profiling refining the plan. The framework supports multiple operators such as SpMV, SpMM, and SpGEMM, adapts to different hardware, and achieves significant speedups over cuSPARSE, including up to 146.61× on certain matrices and 3.39× acceleration for full-batch GCN training.
SearchAtlas is a framework that transforms raw search trajectories of large language model (LLM) agents into structured evidential query graphs, where edges capture how evidence is propagated from queries to the final answer. The automated parsing pipeline achieves a mean edge F1 of 86.0% against human-annotated graphs and remains consistent across repeated runs. Using SearchAtlas, the authors analyze five search agents on three benchmarks, uncovering systematic differences in search scale and evidence aggregation, and revealing process failures such as fragmented answer support, unmet question constraints, and unverified parametric knowledge that correlate strongly with incorrect answers.
TrajFusionNet+ is a transformer-based model that predicts pedestrian crossing intention by fusing sequential trajectory data, visual trajectory overlays, and graph-based scene context. It extends the earlier TrajFusionNet with three attention modules—Sequence, Visual, and Graph—to capture temporal, visual, and relational cues. The model outperforms state‑of‑the‑art methods on the PIE and JAAD datasets and shows better generalization under a joint‑training, separate‑evaluation protocol.
The paper introduces Caption‑once, Frames‑on‑Demand (CFD), a budget‑aware edge‑cloud framework for long‑video understanding. CFD first runs a single offline captioning pass on the edge to build a dual‑track narrative index—an event‑level story skeleton and a clip‑level micro‑log—that is cached for future queries. At query time, a cloud‑side MLLM uses a Visual‑Need Router to decide whether to retrieve keyframes for perceptual questions, thereby limiting visual processing while preserving temporal structure in language space.
The paper introduces the State-Path Tool Menu, a method that presents agents with a short, ordered subset of tools before execution, ensuring that both the final action and its prerequisite tools are available in a usable sequence. By learning a pre‑execution route from the request state to the desired outcome, the menu acts as an execution prior, encoding which tools can run from the current state, how outputs satisfy later inputs, and recurring orders from training paths. Experiments on ToolBench show that this approach boosts online success from 0.737 to 0.898 and outperforms several baselines while covering more complete tool chains with fewer tools.
ContractEval is a diagnostic framework that makes active obligations in procedural instructions explicit by representing them as query‑conditioned obligations. It matches these obligations against response or trace evidence, identifying omissions, wrong branches, ordering errors, extra actions, invariant breaches, and output‑contract violations as distinct conformance failures. In tests on audited procedural contracts, ContractEval detects and localizes all injected structural failures that output‑only and trace‑aware LLM judges miss, though it is not a compliance guarantee and remains calibration‑sensitive.
The paper introduces Cros, a risk‑constrained stopping layer for sequential clinical diagnosis agents that determines when to stop testing and make a diagnosis. Cros combines state‑wise error ranking, policy design on disjoint development splits, and exact tests of selective diagnostic error to provide finite‑sample guarantees. On a MIMIC‑derived abdominal‑pain benchmark, Cros achieves higher state‑error AUROC and lower selective error rates compared to baseline stopping methods, though its performance varies across development resplits.
LexAgentHallu is a new benchmark that profiles hallucinations in legal agents across multi-step interactions. It contains 3,414 instances spanning 17 legal categories and 6 task types, each annotated with a dual-layer taxonomy of 7 high-level and 27 fine-grained hallucination categories. The benchmark introduces fine-grained metrics to quantify and localize failures along an agent’s execution path, revealing patterns such as the Right-Answer-Wrong-Reason effect and clustered hallucination subclasses.
The Era by Eon Benchmark is a new dataset for evaluating large language model agents that interact with enterprise tools. It constructs a complete fictional company with product simulators, internal databases, and benchmark questions, all generated from a shared entity graph to ensure consistency. Exact answer keys are computed from the generated records, allowing precise grading and validation of realism and adversarial robustness across 23 simulated companies.