Semantic Service Level Indicators for AI-Driven Site Reliability Engineering:A Framework for Detecting Infrastructure-Invisible Agent Failures inProduction

Abstract

Traditional Site Reliability Engineering (SRE) observability architectures monitor infrastructure-centric signals: latency, error rate, throughput, and saturation. While effective for traditional software architectures, autonomous AI agents introduce a distinct failure mode: infrastructure-invisible semantic failure. An agent fleet can return HTTP 200 responses, maintain 99.9% platform uptime, and fulfill infrastructure SLOs while simultaneously delivering contextually invalid or corrupted task execution outputs. Standard observability frameworks produce zero telemetry for this class of degradation. This paper presents a production-validated framework consisting of four novel Semantic Service Level Indicators (SLIs) engineered specifically for agentic workflows: Decision Quality Rate (DQR), Tool Invocation Efficiency (TIE), Human Escalation Rate (HER), and Approval Queue Depth Drift (AQDD). Furthermore, we design an Agent-to-Agent (A2A) semantic boundary validation protocol operating as an execution circuit breaker at the semantic layer, alongside an Agent Sprawl governance architecture designed to maintain platform reliability across multi-model, multi-framework enterprise deployments. Empirical evidence from a high-throughput production fintech environment demonstrates that this framework successfully identifies semantic architectural degradations up to 6 hours before downstream transaction failures surface, providing a robust operational foundation for mission-critical AI systems infrastructure.