Why Tenant-Shared Agent Clouds Fail Infra Teams and What to Run Instead

Tenant-shared agent clouds fail infrastructure teams because agents do not behave like the web services those clouds were built to host. They keep state, run for a long time, and use real permissions to do work that cannot be predicted. Production agents need dedicated, isolated execution environments in your own cloud account, with boundaries that contain both resource use and access to data.

Why AI agents are structurally different from the workloads shared infrastructure was built for

Shared-tenant PaaS and multi-tenant Kubernetes were designed around stateless, short-lived services that scale by adding replicas. An agent run can instead last minutes or hours, holding open model sessions, tool connections, and in-process memory. That is a different execution model entirely. A REST handler finishes in milliseconds and releases its request state, while an agent keeps working through a task whose next step depends on what happened before.

That continuing work needs more than a request payload. Agents keep context windows, tool-call histories, memory embeddings, and long-running task queues. Their state belongs to an ongoing task. As a result, adding more workers does not by itself resolve how that state is kept separate or which worker has authority to use it.

Tools make the distinction sharper still. File systems, browsers, code execution, and external APIs give agents the power to act. Their behavior is generated at runtime by a probabilistic system, so generated code must be treated as untrusted. The platform did not review it in advance. Traditional SaaS multi-tenancy largely asks how to keep one tenant’s rows away from another, while agent multi-tenancy extends that question into compute, credentials, inference, and execution.

Three distinct failure modes that emerge when agents run on shared infrastructure

The first failure mode is compute interference. An agent that enters an infinite loop can starve a shared pool of CPU, degrading other customers’ performance immediately. The fault begins with one workload but crosses the tenant boundary because execution resources are shared. Other tenants need not do anything wrong.

The second failure mode is inference leakage. LLM serving systems share KV caches to reuse work, but a cache hit can produce a detectably faster time-to-first-token than a miss. That timing difference can expose information about cached prompts. An NDSS 2025 paper demonstrated prompt reconstruction through this side channel, showing how an adversary could recover another tenant’s prompt token by token. vLLM’s cache_salt parameter limits cached-prefix sharing to requests with the same salt, but it requires deliberate opt-in.

The third failure mode is recycled-state leakage. A pod reused for another tenant can expose workspace files, cached secrets, or in-memory context when cleanup is incomplete. Long-running agents leave a larger residue surface than stateless handlers because they accumulate more working state. Cleanup becomes part of the security boundary. These failures add to each other because keeping compute separate does not protect inference caches, and keeping caches separate does not remove files left for the next tenant.

Why Kubernetes' default security posture makes agent workloads actively dangerous in shared clusters

In a shared Kubernetes cluster, an agent may inherit enough authority to cause harm without exploiting a vulnerability. Through available tools and credentials, it can invoke sensitive APIs, modify files, expose secrets, or consume resources unexpectedly. Where nodes, GPUs, and kernels are shared, the consequences can extend beyond the agent’s own task.

Defaults designed for trusted application workloads do not establish an adequate boundary for untrusted agent execution. A tool call can be valid under the permissions it inherits and still be unsafe for neighboring workloads. Agent isolation therefore has to account for inherited authority as well as obvious attempts to break out.

Kubernetes’ own documentation describes containers as offering “a weaker isolation boundary” than hardware-based virtual machines. Each container on a node still depends on the host kernel, so container separation is not equivalent to a dedicated kernel for each tenant. This distinction sets the scope of containment. OWASP’s LLM application guidance directs teams to treat the model as any other user and validate everything it produces, rather than trusting output because it came from the application’s own model.

Why the shared-infrastructure failure stays invisible until production scale

Single-tenant benchmarks leave out the condition that produces shared-infrastructure failures. They can measure an agent’s behavior in isolation, but they do not reproduce contention from other tenants, eviction risks, or the resulting latency spikes. The workload may run well on its own while remaining unsuitable for a shared production pool.

No hyperscaler has published production cost benchmarks for multi-tenant agent workloads, leaving teams without a public baseline for the economics of this execution model. AWS, Azure, and Google Cloud earn revenue from the compute consumed while customers discover its constraints. That creates an information asymmetry. The customer pays for experimentation without the benefit of the provider’s internal infrastructure knowledge.

The cloud monitoring tools in use were built for deterministic software, but how an agent behaves can change as the decisions it generates change. Operators may see a cost spike or an incident before they can reliably attribute it to a particular execution pattern. The signal arrives after resources have been consumed. Only a small minority of enterprises have successfully scaled agentic AI deployments despite widespread investment intent, and better model output alone cannot resolve the infrastructure gap described here.

The isolation architecture that actually contains agent workloads: microVMs, dedicated kernels, and the silo model

For untrusted execution, each tenant should have at least its own dedicated guest kernel, with hardware virtualization enforcing the boundary. MicroVMs give each tenant that boundary instead of having them all share the same host kernel. A compromise of a tenant’s dedicated guest kernel has a different reach from a compromise of a kernel shared by neighboring tenants.

Firecracker, the microVM technology underlying AWS Lambda, uses Linux Kernel-based Virtual Machine, or KVM, to create microVMs with their own Linux kernels. AWS Bedrock AgentCore uses the same class of isolation for its microVM-based Runtime, placing each user session in a dedicated microVM with isolated CPU, memory, and filesystem resources. The execution boundary follows the user session. These designs show why agent containment belongs below the application’s tenant checks.

Cluster-level tools can make that isolation easier to operate. Google made Agent Sandbox generally available on GKE in May 2026, with isolated network namespaces per agent session, ephemeral filesystem overlays, and automatic scale-to-zero with state checkpointing. Upstream Kubernetes v1.32 came out in December 2024 without Agent Sandbox. The distinction separates a specific platform capability from an assumption that an ordinary cluster already provides it.

Above the execution primitive, the pool, silo, and bridge models describe how tenants share the rest of the system. The pool model uses shared infrastructure with tenant_id filtering, offering low cost but only logical partitioning. It works for workloads whose code is under the platform’s control. The silo model instead gives tenants dedicated environments, aligning the infrastructure boundary with the tenant boundary. The bridge model combines them by isolating regulated tenants while pooling the rest, matching isolation strength to the customer segment’s threat model.

Execution isolation does not replace retrieval isolation. When tenants’ embeddings share one index, a query can return matches from another tenant’s corpus unless strict tenant_id filtering is applied to every query. OWASP LLM Top 10 v2025 introduced LLM08:2025 to address vector and embedding weaknesses, including multi-tenant cross-context leakage. Its guidance also notes that retrieval authorization can be bypassed when retrieval seeds a knowledge graph traversal.

Persistent memory needs an identity boundary as well. Microsoft Azure AI Foundry introduced user-scoped persistent memory in April 2026, using a scope parameter resolved from Microsoft Entra identity or a passed user ID header. Agents use memory that belongs to the authenticated principal. MCP servers are converging on a similar pattern. Together, these layers separate where code runs, what data it retrieves, and whose history it can remember.

Why dedicated, isolated environments in your own cloud account are the right deployment target for production agent workloads

When workloads run in your own cloud account, your team controls the VPC, node groups, IAM boundaries, and network egress rules. That control is the enabling condition for the architecture above. The team can define the execution boundary and the permissions around it rather than depend on a shared service’s undisclosed tenant arrangements. Owning the environment does not by itself make the configuration safe.

A shared-tenant managed service determines which infrastructure is shared, which controls customers can inspect, and where customers can intervene. In a dedicated environment, the team can align those choices with the agent’s threat model. The ability to change isolation and access controls is an operational requirement for making them effective.

Regulated workloads make that requirement harder to defer. SOC 2 and HIPAA controls must be auditable at the infrastructure layer, while cross-tenant access risks demand substantial compensating measures in shared environments. Logs and data stores remain part of that boundary. The DeepSeek database exposure in 2025 illustrates why log exposure, secret redaction, and unauthenticated stores remain central concerns in AI systems. When tenants share those resources, more of their data can be exposed.

Idle GPU time can remain billable even when useful work has stopped, so a dedicated environment needs autoscaling and auto-shutdown to avoid permanently running capacity. Ownership lets the team reduce wasted compute by applying those controls to its own workloads and making spending track actual use more closely. As inference grows, that operating discipline becomes a required part of the deployment model.

What a production-ready agent infrastructure stack looks like on a PaaS that runs in your own cloud account

A PaaS running inside your own AWS, GCP, or Azure account should separate the agent lifecycle into independently scalable services. Those services include ingress, orchestration, scheduling, state and memory, and sandbox execution. Each has a different operational role. Separating them lets the team scale and debug one part without treating the whole agent system as a single deployment. A monolithic agent service obscures those boundaries as load grows.

The sandbox tier is where execution isolation becomes concrete. Any service running AI-generated code should use microVMs or gVisor-isolated containers. Those options should not be treated as identical. Where the requirement is a dedicated guest kernel, microVMs provide the primitive described above. The platform must make the chosen boundary explicit rather than hide it behind a generic claim that workloads are containerized.

Inference and training need their own capacity controls. Autoscaling GPU node groups with auto-shutdown on idle should be platform behavior. Otherwise, idle clusters turn temporary demand into continuing waste. This is where a PaaS can remove operational work without removing infrastructure control. The team retains ownership of the environment while the platform handles the routine enforcement of its resource policies.

Security controls should follow the same approach. Network segmentation, secret management, audit logging, and CVE patching should be platform primitives rather than engineering projects postponed until after launch. They support the SOC 2 and HIPAA controls that apply to agent workloads without making compliance a substitute for containment.