Managed AI Services vs Self-Hosted Agent Runtimes

Choosing between managed AI services and self-hosted agent runtimes is a decision about who controls the system and who does the work. It is not a binary choice. The right fit depends on compliance, operational capacity, utilization, and governance, with several workable positions between full delegation and full ownership.

Why agent infrastructure became a real decision in 2026

Production agents outgrew what a framework alone could provide. Durable state, sandboxed execution, credential management, and audit trails became requirements for shipping a product. A framework could express the agent’s logic without supplying the machinery needed to operate it safely. That gap became an architectural decision in 2026.

By spring 2026, productized runtimes on both the managed and self-hosted paths had matured to address that gap directly. Teams could delegate the runtime or take responsibility for it themselves, rather than assemble every supporting service from scratch. But either choice assigned real obligations. For any team shipping agents to real users, the question had become unavoidable.

What the market actually looks like — four distinct categories, not two

“Agent hosting” covers four different kinds of platform. The first is a managed runtime with a built-in harness, such as Claude Managed Agents or the AgentCore harness. The team supplies prompts and tools, while the platform runs the agent loop, maintains state, and handles wake-ups. The second is a bring-your-own-agent runtime, where the team writes the loop and deploys it while the platform handles sessions, isolation, and scaling. AWS AgentCore Runtime and Google Agent Runtime belong in this category, as do self-hosted runtime deployments built around LangGraph.

The third category is a durable execution engine. Temporal, Inngest, Trigger.dev, and Restate run agent logic as workflow steps with checkpoints and retries, particularly when the agent sits inside a larger backend. The fourth is raw compute and sandboxes, where the team owns everything above the machine.

A framework is a different thing again. LangGraph or CrewAI provides code for building an agent, not, by itself, the runtime that hosts it. A self-hosted deployment built around a framework still needs that operational layer.

What every production agent actually needs from its runtime

Agents are long-running, stateful, and able to take consequential actions. Their hosting requirements follow from those properties. Process supervision must restart an agent after a crash without losing its position in a task. Durable state must preserve history, files, and pending work across deploys and restarts. Restarting a process is not enough if its work disappears.

Execution also needs a boundary around it. Tool calls and generated code must be isolated so that one agent cannot reach another tenant’s data or the host system. Credentials must be passed in at runtime and kept out of the image or prompt. These are shared requirements, whether a vendor operates the runtime or the team does.

Governance needs more than a generic execution log. An action-level audit trail should make agent activity attributable and reviewable, with immutability providing stronger protection for the record. A shell invocation in a CI log is not an adequate substitute. Idle handling also belongs in the evaluation because agents often wait between turns, and billing for those waits can distort costs. Most platforms handle the first few requirements readily, leaving governance, compliance, and cost at scale as the sharper differentiators.

Where managed runtimes genuinely win

For teams without dedicated platform engineers, a managed runtime with a built-in harness can compress months of infrastructure work into days. Prebuilt connectors and no infrastructure provisioning can put a first agent into production in days to weeks. It takes much longer to set up a self-hosted deployment with enterprise-grade observability and security hardening. The included sandbox, checkpointing, credential vault, and tracing are not minor conveniences. Each would otherwise be a separate engineering project.

Claude Managed Agents launched April 8, 2026, with sandboxed code execution, checkpointing, credential management, scoped permissions, and end-to-end tracing. Its pricing combines standard token rates with $0.08 per active session-hour. Only active runtime is measured to the millisecond, which suits workloads with gaps between turns. Google Agent Runtime likewise excludes idle time between turns, while charging per vCPU-hour and per unit of memory per hour.

AWS Bedrock AgentCore harness became generally available on June 17, 2026. It charges for AgentCore resources used, including vCPU-hour and GB-hour, without a separate harness fee. Gateway, Memory, Identity, Policy, and Observability are billed separately. Sessions receive a microVM with an isolated filesystem, but persistent filesystem storage is an opt-in preview feature. Isolation and persistence are separate capabilities here.

AgentCore supports Bedrock, OpenAI, Gemini, and any provider compatible with its model-routing interface, with switching possible between turns in the same session. Claude Managed Agents is scoped to Claude. Managed autoscaling also removes the need for the team to write scheduling logic, but vendor concurrency limits or credit exhaustion can become binding constraints. Managed works well for a team with little time or staff to run the system, as long as the product can work within those limits.

Where self-hosted runtimes become the only viable path

If regulatory, contractual, or internal-security rules prohibit code, data, or secrets from leaving the team’s infrastructure, a vendor-hosted runtime cannot satisfy that boundary. Anthropic offers self-hosted sandboxes for Claude Managed Agents, but orchestration and the model remain on Anthropic’s cloud. Anthropic states that the service, including self-hosted sandboxes, is not eligible for Zero Data Retention or HIPAA business associate agreement coverage.

Control requirements also need careful interpretation. SOC 2 CC8.1 concerns change management, while HIPAA §164.308(a)(3) addresses workforce security and appropriate access to ePHI. The latter does not spell out a rule that change authors and reviewers must be different people. Even so, an agent that opens, approves, and merges its own PR presents a governance problem when independent approval is required. Self-hosting provides control over the instrumentation used to enforce and demonstrate that separation, but it does not make separation automatic.

SOC 2 CC6.1/CC7.2 and HIPAA §164.312(b) are relevant to access and activity monitoring, but the HIPAA provision does not explicitly mandate immutable, action-level logs, which are recommended as best practice for a stronger record. Many managed runtimes do not expose action-level logs to the customer, which can block a team from meeting its own audit requirements.

Concurrency creates a separate reason to own the runtime. Self-hosted deployments are bounded by the customer’s cluster rather than vendor-imposed concurrency caps or credit exhaustion. Self-hosting gives the team control over where the capacity limit sits and how capacity is added. At high, predictable utilization, that control can also improve the economics.

How cost actually scales on each path

DIY comparisons often leave out patching, monitoring for silently failed jobs, and on-call responsibility. Those engineering hours are part of production cost, even when they do not appear on an infrastructure invoice. Managed token and session-hour pricing is cheaper for most teams below a high request-volume threshold. Above it, per-unit economics can favor self-hosting, but the notes provide no universal break-even number.

The break-even threshold for self-hosting at the model layer is well above what most early-stage teams consume. Below that point, managed APIs usually cost less overall once the cost of engineering work is counted. Owning the agent runtime and owning inference are distinct decisions with different operating burdens.

For teams running inference at scale, compute pricing deserves its own scrutiny. Specialized GPU providers offer H100 hardware at meaningfully lower on-demand rates than hyperscalers, in exchange for fewer managed services and a smaller geographic footprint. The same control-versus-convenience tradeoff appears beneath the runtime. Production infrastructure in the team’s own AWS, GCP, or Azure account offers another point on that spectrum, where the team can see costs directly and the platform runs the control plane. That skips the shared-tenant PaaS billing layer, and the team does not have to manage every part of the infrastructure.

The five questions that place a team on the spectrum

The practical decision starts with five questions. First, do regulatory, contractual, or internal-security requirements prohibit code, data, or prompts from leaving the team’s infrastructure? A yes makes self-hosting a requirement, regardless of the other answers.

Second, does the team have a platform engineer or equivalent who can own runtime operations, incident response, and security hardening? Without that person, self-hosting’s hidden cost is likely to arrive as production incidents. If the team lacks that capacity, the decision needs to account for who will supply it.

Third, are workloads predictable and highly utilized, or bursty and intermittent? Predictable demand favors self-hosting’s economics, while intermittent work favors managed services that do not bill idle time. Fourth, does the product need model portability or access to several providers? A runtime tied to one model family leaves the team dependent on that family as the choice of models changes. Neither answer settles compliance, but both shape which permitted option is practical.

Fifth, does the product need action-level audit logs, role-based access to its agent runs, and human-in-the-loop approvals? The team needs to verify that a managed runtime exposes those controls to the customer rather than assume an internal capability is available externally. Governance must be usable by the team. A hybrid can be legitimate, with the harness managed and data processing, background workers, and persistent storage in the team’s own cloud account. That split can satisfy some data-residency requirements, but not a blanket prohibition on external orchestration.

These answers place a team between delegation and ownership. Some constraints rule out options, while the remaining answers determine how much operational responsibility is sensible. For teams whose answer to operational capacity is “not yet” but whose answer to compliance restrictions is “soon,” the right move is a platform that runs in their own cloud account today. The compliance transition then need not become a migration.

What the infrastructure layer under the runtime should look like

The runtime decision depends on the infrastructure beneath it. A self-hosted runtime on shared-tenant PaaS infrastructure inherits that platform’s compliance and reliability limitations. Calling the runtime self-hosted does not establish control over the compute layer. Data residency and audit advantages depend on where that compute runs and who controls it.

Running agents in the team’s own AWS, GCP, or Azure account provides cost transparency and control over compliance scope. It also lets agent compute sit near the data stores it accesses, reducing latency and egress cost. Cluster management, CVE patching, autoscaling, and networking consume platform engineering time without creating product value. A platform that handles those tasks while keeping compute in the customer’s account separates infrastructure ownership from routine operation.

GPU workloads need the same discipline as agent runtimes. With H100 instances in its own cloud account, the team can use reserved-instance pricing, control data residency, and keep compute in the same place as training data. The deployment layer should also encode relevant SOC 2 and HIPAA controls as infrastructure and collect evidence continuously. That makes compliance scope more tractable for a small team rather than turning it into a multi-month infrastructure project.