Hugging Face Trending Papers

Governing Enterprise AI Traffic at Scale: Design and Operation of a Policy-Driven, Multi-Vendor LLM Gateway on Envoy

arXiv AI
Oct 27

Governing Enterprise AI Traffic at Scale: Design and Operation of a Policy-Driven, Multi-Vendor LLM Gateway on Envoy

The paper describes the Cosmos LLM Gateway, a policy‑driven, multi‑vendor gateway built on Envoy that manages enterprise LLM traffic at PayPal. It separates latency‑sensitive control decisions into an external service while handling wire‑format translation and token metering in‑proxy via a Rust WebAssembly filter, ensuring minimal overhead. Governance is declarative, managed through Open Policy Agent, and includes features like data‑loss‑prevention, budget scopes, and a control plane for role‑scoped authoring and approvals. The gateway currently serves over 12,000 developers, processes roughly 200 billion tokens per day across 100+ models, and maintains 99.999% availability with only a few milliseconds of added latency.

By Mithun Sasidharan, Srinivasan Manoharan, Rupesh Thakur, Abhinav TK, Ashwin Hegde, Anand Gautam
arXiv AI
5d ago

Cross-Interface Authorization for AI Computer-Use Agents

The paper introduces a protocol for cross‑interface authorization of AI computer‑use agents, ensuring that policy decisions remain consistent across graphical interfaces, command lines, APIs, scripts, and Model Context Protocol servers. It defines a profile‑bounded system that enumerates all effect‑bearing routes, normalizes requests to canonical effects, issues prepare tickets, and re‑verifies all relevant state at commit time, requiring a sink receipt for a committed result. Extensive testing across two runtime mappings produced thousands of traces, with rigorous checks confirming no semantic regressions, mutant rejections, and correct outcome matching.

By Genliang Zhu, Chu Wang
arXiv AI
Jun 15

STREAM: Multi-Tier LLM Inference Middleware with Dual-Channel HPC Token Streaming

arXiv:2606. 13968v1 Announce Type: cross Abstract: Researchers and practitioners working with large language models face a fragmented landscape: local models are free and private but hardware limits the model size and context windows a researcher can use; institutional HPC centers offer powerful GPU resources at no marginal cost and keep data within institutional boundaries, but operate behind firewalls and are designed for batch jobs rather than interactive use; commercial cloud APIs provide frontier-model quality on demand but impose significant cost and data retention policies unsuitable for sensitive research data.

By Anas Nassar, Steve Mohr, Leonard Apanasevich, Himanshu Sharma
arXiv AI
Aug 19

PACE: Policy-Attested Contract Execution for Safe AI Agents in Decentralized Finance

The paper introduces PACE (Policy‑Attested Contract Execution), a framework that sits between large‑language‑model (LLM) based autonomous AI agents and on‑chain DeFi operations. PACE defines typed transaction intents, a deterministic policy verifier, and signed Policy Decision Records (PDRs) that cryptographically bind an approved intent, policy, and simulation report to the exact on‑chain execution bytes, providing replay and expiration protection. In evaluations across 40 tasks and six baselines, PACE achieves zero unsafe executions and zero false positives, outperforming unguarded agents by a large margin.

By Rabimba Karanjai (Larry), Yang Lu (Larry), Richard Williamson (Larry), Hemanth Hm (Larry), Prakhar Mehrotra (Larry), Lei Xu (Larry), Weidong (Larry), Shi
arXiv AI
Oct 26

MCPHunt: An Evaluation Framework for Cross-Boundary Data Propagation in Multi-Server MCP Agents

MCPHunt is a benchmark framework that isolates non‑adversarial credential propagation across multi‑server MCP trust boundaries. It uses canary‑based taint tracking, a controlled coverage design with risky, benign, and hard‑negative conditions, and completion‑requires‑secret stratification to separate faithful from policy‑violating propagation. Across 3,615 traces from 5 models, policy‑violating propagation rates range from 11.5 % to 41.3 %, with mitigation via prompting reducing rates to 0.3 % while retaining 80.5 % utility.

By Haonan Li, Tianjun Sun, Yongqing Wang, Qisheng Zhang
arXiv AI
Jul 21

Deterministic Replay for AI Agent Systems

arXiv:2607. 16200v1 Announce Type: new Abstract: AI agent systems that couple large language models (LLMs) with external tools and APIs are inherently non-deterministic: LLM sampling variance, external API state, CDN infrastructure headers, and execution-environment noise collectively prevent any prior agent run from being faithfully re-executed.

By Rasheed Mudasiru
arXiv AI
Jun 2

AgentRedBench: Dynamic Redteaming and Integration-Aware Defense for LLM Agents over SaaS Integrations

arXiv:2606. 02240v1 Announce Type: cross Abstract: Indirect prompt injection in tool-use agents is a concrete production threat: LLM agents read from integrations (third-party services such as Gmail, Salesforce, or Jira accessed through tool calls) whose response content the user neither writes nor controls.

By Hiskias Dingeto, Will Leeney