Agent Harness: The Boundary Between Model and Platform
A model produces outputs and can request actions. On its own, though, it neither executes client-side tools nor operates the system around them. The Agent Harness is the software and control layer that organizes model calls and connects them to tool execution, state, memory, permissions, guardrails, and operational services. This boundary is a practical architectural model, not a universal standard. Its exact shape depends on the framework, provider, and deployment.
An agent's behavior therefore depends on more than its model. The harness determines how a request moves through the system, which component may act on it, what context persists, and which operational controls surround execution. So the useful security question is not “model or harness?” It is “which component owns each decision and action?”
What an Agent Harness Is — and What the Term Does Not Standardize
“Agent Harness” is useful shorthand for the scaffolding that turns model output into an operating agent system. One published definition places tools, memory, sandboxes, an orchestration loop, guardrails, and permissions around the model. It captures the concept well, but remains a vendor definition rather than a normative specification.
The wider vocabulary is unsettled as well. Anthropic, for example, distinguishes workflows that follow predefined code paths from agents that dynamically direct processes and tool use. It describes an augmented model with retrieval, tools, and memory as a basic building block. Other frameworks may call overlapping parts of the arrangement a runner, runtime, SDK, control plane, or agent server.
In this article, the Agent Harness covers the control flow around model calls and the components it owns or integrates for tools, state, memory, permissions, guardrails, and execution. The model remains responsible for generating its output within the interaction. This harness boundary does not imply a particular implementation, deployment model, or security guarantee.
The Control Flow That Separates Model Output from Harness Action
The boundary is easiest to see when a model requests a tool. In Anthropic's documented tool-use flow, the model returns a structured tool call. A client tool then runs in the application, and its result is sent back for a later model response. A server tool instead runs on provider infrastructure. The request is model output; execution belongs to the client application or the provider hosting the tool.
A runner or application loop coordinates the work around that request. OpenAI's Agents SDK runner, for example, loops over model calls and their outputs, executes tool calls, appends results, handles handoffs, and calls the model again. Google's Agent Development Kit documents a similar separation: its runner processes events and actions, including state changes through services, before agent logic resumes.
Even control of the loop can be assigned in different ways. OpenAI documents an SDK-managed loop as well as an application-owned loop using its Responses API. So “the harness owns orchestration” means that a named software component or operating team owns it in the chosen architecture—not that every framework puts it in the same package.
A concise boundary model is:
Input and context → model output or tool request → harness routing and policy → tool execution → result and state update → next model call or final output
This is a conceptual map rather than an implementation procedure. At every arrow, responsibility may pass between the application, an agent framework, an execution environment, and a model or tool provider.
Four Operating Responsibilities the Model Does Not Own
Persistence
Conversation or task continuity needs a system outside the model call to retrieve and store state. OpenAI's session documentation describes a runner that loads session history before a run, then stores new inputs, responses, and tool calls afterward. LangGraph distinguishes checkpoints for graph state from stores for application-defined data across threads. It also notes that in-memory checkpointing does not survive a process restart.
Persistence is therefore a harness or platform responsibility. That layer determines what state exists, where it is stored, how long it remains, and which later run may retrieve it. Model context is input assembled for a call, not a durable storage system.
Scaling
Serving requests and executing agent work can follow different scaling paths. LangGraph's Agent Server documentation separates API servers, which handle client requests, from queue workers, which execute agent code. Its scaling guidance links request concurrency to API replicas and run concurrency to worker capacity. Excess work waits; too many jobs per worker can increase latency and memory use.
The operating platform must provision and govern admission capacity and execution capacity separately. A different model will not fix a saturated worker pool or an undersized request tier.
Observability
An agent system is observable only when its components emit useful telemetry. OpenTelemetry identifies traces, metrics, and logs as outputs of instrumentation. It also notes that code instrumentation provides application semantics that zero-code methods may miss. The OpenTelemetry Collector can receive, process, and export telemetry with batching, retries, encryption, and filtering, but it cannot supply application meaning that was never emitted.
The harness and platform must add useful semantics to model calls, tool requests, execution, state transitions, and failures. A model alone does not provide end-to-end operational visibility.
Resource governance
CPU and memory controls are enforced below the model. In Kubernetes, the scheduler uses resource requests for placement. The kubelet and container runtime apply limits, which the kernel ultimately enforces through cgroups.
A harness may request or select resources, but the execution platform enforces them. Keeping those roles distinct separates a policy declaration from the mechanism that makes it effective.
Managed, Self-Operated, and Sovereign Describe Different Boundaries
These labels address different questions. “Managed” concerns who operates service components. “Self-operated” identifies the components an organization runs itself. “Sovereign” concerns control and exposure across several dimensions. None of these labels, on its own, reveals who can access data, who holds cryptographic control, or who owns every operational task.
| Description | Responsibility boundary to inspect | What the label alone does not establish |
|---|---|---|
| Managed service | Which layers the provider operates, and which data, identity, encryption, and integration duties remain with the customer | That the customer has no security responsibilities |
| Self-operated components | Which harness, storage, telemetry, and execution components the organization deploys and runs | That every dependency is organization-owned or on premises |
| Sovereign posture | Jurisdiction, data and AI controls, operations, supply chain, technology, security, access visibility, cryptographic control, and operational autonomy | A single binary property of the deployment |
AWS's shared-responsibility guidance makes the first limitation explicit. Even with abstracted services, customers retain duties such as data management, encryption choices, asset classification, and IAM permissions. NIST's cloud definition shows why location and operation must also remain separate: a private cloud can be on or off premises and operated by the organization, a third party, or both. The European Commission's Cloud Sovereignty Framework treats sovereignty as multidimensional, including compelled access, customer cryptographic control, access visibility, and operational autonomy among its concerns.
A reliable comparison is concrete. Name the component, operator, data path, access path, and control owner. Marketing labels may summarize that arrangement, but they cannot replace it.
Where Harness Ends and Existing Runtime, Sandbox, and Security Guidance Begins
Harness, runtime, and sandbox are neighboring concepts, not synonyms. The harness organizes the managed loop and surrounding behavior. A runtime coordinates the interaction flow, state, tools, and services the agent needs while operating. A sandbox is the bounded surface in which selected code or tools execute. Containers and microVMs are possible isolation mechanisms.
This boundary also keeps “Agent Harness” from becoming a catch-all security term. Detailed controls for isolation, egress, credentials, and host integration belong in a threat model for AI agent security across the path from instruction to action. The distinction between coordination and bounded execution is developed further in agent runtime versus agent sandbox.
When assigning responsibility, ask where each function actually lives:
- Does the model produce a proposal, or does another component authorize and execute it?
- Which component stores state, and which identities can retrieve it?
- Who operates request handling, worker capacity, telemetry, and resource enforcement?
- Which provider or operator can access prompts, results, credentials, and stored data?
These questions preserve the model–harness boundary without suggesting that the harness is always more important than the model. They also survive changes in framework terminology: responsibility follows control, execution, state, operation, and access—not the label attached to the system.