How to evaluate and approve AI agent infrastructure

Approve an infrastructure foundation for autonomous agents only if it passes four independent gates. The execution model must fit the measured workload. Every state class needs acceptable loss and recovery guarantees. Every network path must be controlled. And the team must be able to carry the operating responsibilities. Reject an option that crosses a hard technical or policy boundary; defer when decision-relevant evidence is missing. Strength in one gate cannot offset failure in another.

This is not a verdict on cloud infrastructure in general. It is a test of one candidate foundation: the compute services, persistent stores, network paths, and operating model proposed for a defined autonomous-agent workload. Runtime orchestration and sandbox execution are adjacent control surfaces. Teams that need to draw those boundaries first can distinguish a cloud development environment from an agent workspace and distinguish an AI agent runtime from an agent sandbox.

What exactly is being approved?

Before scoring the candidate, capture it in an architecture decision record. Name the workload, execution location, persistent state, ingress, egress, external tools, data locations, and operator for each component. Label every responsibility as customer, provider, or shared. If any platform boundary is unnamed, defer: an incomplete option cannot be approved.

Assess the option across four boundaries:

• Execution: where agent processes and jobs run, with which resource and runtime limits.
• State: where checkpoints, long-term memory, artifacts, and external side effects persist.
• Data flow: how requests, tool calls, model calls, and stored data enter, leave, and cross jurisdictions.
• Operations: who owns scaling, observability, upgrades, backup and recovery, incidents, cost control, and exit.

Do not treat a managed or self-hosted label as the decision. A deployment can place its control plane, agent servers, and data plane with different operators and in different locations. Those placements reveal responsibility and residency more precisely than the label does (LangSmith deployment documentation).

Compute and resources: Does the execution model fit the workload?

Approve the compute gate only when measured P95 CPU and memory, accelerator needs, kernel or image constraints, concurrency, start-time targets, and connection duration fit current documented limits. If representative measurements cannot establish production behavior, require a load test.

A managed container path can fit isolated batch work without making the platform team operate servers, clusters, scaling, server selection, or cluster packing. AWS Batch on Fargate also gives each job its own isolation boundary (AWS Batch documentation). This is a narrow, documented fit—not a general endorsement.

For the current AWS path, move the candidate away from Fargate and assess EC2 when a job needs more than 32 vCPUs, more than 244 GiB of memory, a GPU, custom AMI or Linux parameters, or concurrency beyond Fargate throttling limits (AWS guidance). Any one of these requirements excludes that compute option.

Start time is a trade-off, not a promise. AWS reports roughly 30 seconds to provision Fargate jobs versus a few minutes for a new EC2 instance, and notes that image size and job count affect the result (AWS guidance). Test the actual workload instead of adopting that vendor observation as an SLO.

Long-lived connections need a separate gate. Cloud Run supports WebSockets only up to the configured request timeout, with a documented maximum of 60 minutes. Clients need reconnection handling. Because autoscaling can route later requests elsewhere, state must also be synchronized outside an instance (Google Cloud Run documentation). Reject a stateless serverless option if the workload requires an uninterrupted connection beyond its limit and cannot reconnect safely.

Compute decision threshold: approve only if every measured requirement falls within a documented boundary and the load test meets the candidate SLO. Reject an incompatible hard limit. Defer when measurements are missing, service limits are stale, or a test remains unresolved.

Storage and persistence: Which state may be lost?

Start with a state inventory. Separate thread checkpoints, cross-thread long-term state, generated artifacts, and external side effects. LangGraph, for example, distinguishes graph-state checkpoints from application-defined, cross-thread store data (LangGraph persistence documentation). Each class can require a different recovery objective, consistency model, retention period, and deletion behavior.

A checkpoint does not make an external action exactly once. LangGraph can resume from a successful super-step boundary without rerunning successful nodes from that step (LangGraph checkpointer documentation). The application still needs idempotency or reconciliation for an API call, payment, message, or other side effect whose outcome may outlive the checkpoint.

Storage lifecycle is another explicit choice. Kubernetes persistent volumes survive individual Pods, while claims specify capacity and access modes. The reclaim policy determines whether storage is retained or deleted (Kubernetes documentation). Approve the settings only when they match the state class; the word “persistent” is not enough.

Durability has a hard boundary for irreversible work. PostgreSQL synchronous commit waits for write-ahead log records to reach durable storage before reporting success. Asynchronous commit can reduce latency but may lose recently acknowledged transactions, and PostgreSQL warns against it when clients take external actions that rely on the transaction being remembered (PostgreSQL documentation). Reject a design that acknowledges durable intent asynchronously before triggering an irreversible downstream action.

Consistency also creates a trade-off. PostgreSQL's Serializable isolation prevents anomalies beyond Read Committed by producing an effect equivalent to serial execution. Applications must handle rollbacks and retries, and may incur a runtime cost (PostgreSQL documentation). The approval record should identify acceptable anomalies and explain how retries affect agent actions.

Do not collapse regional durability and cross-region recovery into one guarantee. Google Cloud Storage confirms writes after redundant storage, then replicates across regions asynchronously. The replication objective depends on the selected mode (Google Cloud Storage documentation). Record the accepted recovery point objective for each state class instead of inferring it from a regional label.

Storage decision threshold: approve only when every state class has accepted RPO, RTO, consistency, retention, deletion, replay, and side-effect behavior. Reject the option if lossy acknowledgement can precede an irreversible action or a required guarantee cannot be configured. Defer when configuration evidence or recovery tests are missing.

Networking and data flow: Can every path be controlled?

Map ingress and egress separately. Private ingress does not imply private egress: Microsoft documents a configuration in which callers use a private endpoint while agent egress remains public (Microsoft Foundry networking documentation). For each path, name the source, destination, protocol, identity, routing control, firewall or policy enforcement, operator, and data class.

A private runtime connection can still require an explicit internet route. AWS AgentCore can connect to private VPC resources and accept private API calls through PrivateLink. Its documented VPC path, however, requires a private-subnet route to a NAT gateway for internet access; a public subnet alone does not provide that access (AWS AgentCore documentation). Approve only routes declared in the architecture record.

Routing ownership and data-resource ownership are separate gates. In Azure's bring-your-own virtual network model, the customer controls address ranges, firewall, peering, and routing, while the deployment may use platform-managed or customer-owned data resources (Microsoft Foundry networking documentation). The network diagram therefore needs both the path operator and the data owner.

Tool support can be a hard exclusion. Microsoft states that some agent tools do not support network isolation and that some reach destinations over the public internet (Microsoft Foundry networking documentation). Reject a complete-isolation claim if a required tool cannot operate in the isolated mode. If the current support matrix has not been checked, defer.

Kubernetes network objects are not evidence of enforcement by themselves. Kubernetes defines Service, Gateway or Ingress, and NetworkPolicy APIs, but the CNI and other implementations provide much of the behavior. A NetworkPolicy can have no effect when the network implementation does not support it (Kubernetes networking documentation). Require an effective-policy test, not just a manifest review.

Gate storage location and processing location separately. Google documents that data at rest can remain in a selected location while requests sent to a global endpoint may be processed elsewhere without a data-residency guarantee (Google data-residency documentation). A regional storage setting does not prove regional processing.

Network decision threshold: approve only when every ingress, egress, model, tool, and storage path is supported, controlled, tested, and assigned to an operator. Reject a prohibited route or unsupported isolation requirement. Defer when any destination, location, implementation, or current tool capability is unknown.

Operating model and sovereignty: Who carries which burden?

Managed infrastructure shifts tasks; it does not erase them. In Google Kubernetes Engine's shared-responsibility model, Google handles underlying infrastructure and patches node components and the control plane. Customers remain responsible for workloads, images, data, IAM, monitoring, alerts, and incident response (GKE documentation). Make sure the candidate ownership matrix covers both sides.

A self-operated Kubernetes option carries a larger platform burden, including certificates, load balancing, separate and backed-up etcd, upgrades, nodes, and capacity (Kubernetes production documentation). Reject this operating model if the team cannot meet the required availability and recovery objectives with named owners and tested procedures.

Durable execution has its own operating and cost dimensions. Temporal's self-hosting checklist covers scale, reliability, monitoring, visibility storage, sequential upgrades, archival, and disaster recovery (Temporal self-hosted guide). Its managed cloud pricing meters actions, active workflow state, and retained history storage separately (Temporal Cloud pricing). Model all three cost drivers against the expected workload and retention instead of relying on a single request count.

Observability must cover the failure modes in the decision. Kubernetes' Metrics API intentionally provides resource metrics for autoscaling and basic inspection; it does not replace a pipeline for metrics, logs, and traces (Kubernetes observability documentation). Defer until the team can detect and investigate failures in execution, persistence, networking, and operations.

Sovereignty is a route-level assessment, not a provider label. Evaluate the laws and access conditions governing compute, model endpoints, operators, infrastructure, and stored data. The BSI recommends objective, verifiable, risk-based criteria and notes that shared responsibility limits customer control (BSI cloud autonomy criteria). For a practical decomposition, assess compute, model, and operator jurisdiction.

Exit and third-country access need distinct evidence. The EU Data Act requires covered data-processing services to address switching and portability, disclose infrastructure jurisdiction, and provide safeguards against conflicting third-country governmental access to non-personal data held in the Union (EU Data Act, Articles 23, 28, and 32). Counsel should determine applicability and required safeguards for the candidate. The infrastructure decision record should capture the resulting controls, export format, owner, and exit test.

Operating-model decision threshold: approve only when responsibility, capacity, SLOs, observability, cost, jurisdiction, access, and exit are accepted and evidenced. Reject if the team cannot carry a required responsibility or meet a mandatory access boundary. Defer while ownership, cost, legal applicability, or an exit test remains unresolved.

Decision: approve, reject, or defer

Use the matrix below as the signed decision record. Do not average the gates.

Apply a strict decision rule:

• Approve only when all four gates pass with current evidence.
• Reject when any hard exclusion applies.
• Defer when no hard exclusion has been established but any decision-relevant evidence is missing.

For a real proposal without attached measurements, configuration exports, path tests, recovery tests, a responsibility matrix, a cost model, a jurisdiction assessment, and exit evidence, the only supportable result is defer. Give each missing item an owner and due date. Record any approval conditions, the decision owner, and a review date. At that review, recheck product limits, supported network modes and tools, pricing, responsibility boundaries, and applicable switching or third-country-access rules.