Deploying a Private AI App Generator in a Fully Air-Gapped Cloud Account for Internal Business Applications

Deploying a private AI app generator in a fully air-gapped cloud account means more than keeping its interface private. Model weights, runtimes, dependencies, and networking must form one self-contained artifact with no outside calls. For internal business applications, that is the core design decision. It determines what can run, how updates enter, and whether the deployment meets the data rules that justified isolation.

Why a private cloud account is not an air-gapped deployment

A team sets up a private VPC, puts its application inside it, and assumes the isolation work is done. The application has a private address. But a VPC uses software-defined networking in a shared public cloud, and the provider’s IAM control plane, billing telemetry, and administrative channels can still be reached from the public internet. Those connections distinguish a private account from a disconnected environment. The account boundary alone does not establish where prompts, outputs, or weights go.

A true air gap has no physical WAN connections, external DNS resolution, or outbound telemetry. New data enters through physical transport media or unidirectional data diodes. A code assistant that validates its license remotely is not air-gapped, nor is an inference engine that sends anonymous usage metrics. Both create a path to the outside even if they never send a prompt.

The location of inference is therefore more revealing than the location of the interface. A chat application in a VPC that calls a public model API still sends its prompts outside. AWS PrivateLink keeps traffic off the public internet, but inference still runs in the provider’s service. Private transport is not local execution. Where data rules require weights and compute to remain under the organization’s control, a private endpoint does not meet that requirement.

Which regulated use cases actually require the full air gap

For most regulated firms, private cloud is the correct starting point. Full isolation belongs where the application’s data already lives on an isolated network or where any external path is legally untenable. Choose the least isolated mode the data rules allow.

Classified workloads and some defense, government, and healthcare deployments meet that test, but a framework’s name alone does not settle it. CMMC Level 2 requires controlling and limiting external connections, while ITAR permits certain encrypted external paths for unclassified data. HIPAA presents a different reason for isolation. When PHI never leaves the organization’s environment and the AI vendor receives none, there is no PHI disclosure to that vendor as a business associate. That is structurally simpler than evaluating a cloud AI service that receives patient records, even under a BAA.

Treating the full software stack as a single self-contained artifact

The useful mental model is an immutable artifact. That artifact includes model weights, container runtimes, tokenizers, system libraries, and every dependency required to execute. Nothing arrives on demand from the internet. A runtime request to Docker Hub, PyPI, a license server, or a telemetry endpoint means the package was never self-contained.

This changes what counts as a deployable release. The model is only one component of it, and local weights do not compensate for a missing library or an agent that calls an outside service. Every piece must be in place and checked before the network closes. Integrity must also survive each transfer boundary. If the supply chain cannot prove the hash of each component at ingress, it cannot establish that the installed release matches the validated one.

Immutability means updates replace the packages in a controlled way, so changes to packages that are running cannot go untracked. Three independent stacks must operate locally without outbound calls. They are orchestration, including the agent loop in LangGraph or ReAct, memory, including the dependency graph and RAG corpus in Memgraph or Weaviate, and inference, served through tools such as Ollama or vLLM.

Offline dependency management, internal registries, package mirrors, and data transfer procedures

To put that model into practice, provision the internal sources before closing the air gap. Start with a customer-internal registry for every OCI image the deployment needs. On a connected jumpbox, pre-pull and scan the images, then move them through the controlled transfer process into that registry. Public pulls are not a fallback option. Inside the perimeter, containers must not fetch images from Docker Hub, Quay, or another public registry.

Next, provide an internal Artifactory or equivalent package mirror for pip, apt, and yum requests. Pin package versions so that the release depends on a known set of files rather than whatever an upstream repository serves later. The mirror itself needs controlled offline updates. Live synchronization would reopen the external path the architecture is meant to remove. Model weights must go through the same process, starting with a download on the connected jumpbox.

Hash checkpoint files using SHA-256 at minimum, then transfer them on encrypted storage media or through a data diode. Write-once optical media can also carry the files. Verify the hashes at ingress before the inference server loads the weights. The environment also needs its own DNS root zones and must have no external domain resolution. Attempts to resolve external hostnames should fail silently and be logged as potential policy violations, because a bundled component may still expect an outside service even when all its files are local.

Inference stack selection and model-hardware co-optimization inside a disconnected environment

Select the model, serving framework, and hardware against peak concurrency, memory needs, and token throughput as one system. A disconnected deployment has no external model fallback or burst capacity to rescue a poor fit. Procurement fixes much of the available headroom. The hardware plan should therefore make room for next year’s model as well as the model that fits today, while the serving framework must work under the same offline constraints.

vLLM and Ollama are established options for air-gapped inference, with TensorRT-LLM inside an OCI host offering another route when NVIDIA hardware is the base. Quantization changes the trade-off between memory and throughput. vLLM supports FP8 quantization, which can reduce memory use and increase token throughput without production-breaking accuracy loss for most internal business application workloads.

NVIDIA L40S nodes are inference-optimized and cost substantially less than H100 systems. H100 SXM5 nodes offer higher throughput for large models and multi-tenant workloads, while H200 systems provide larger HBM3e memory per card for large-parameter models and large KV caches. Multi-node inference also requires high-speed InfiniBand or RoCEv2 spine-leaf networking. The required networking represents a significant capital expense beyond a routine Ethernet upgrade.

Running costs extend beyond the cards themselves. An 8-GPU H100 node draws substantial power, and electricity, cooling, and PUE overhead need separate budgets. At sustained high utilization, purchased hardware can break even against hyperscale rentals within months to a couple of years, depending on rental terms, with significant savings over a three-year lifecycle. At lower utilization, cloud rental remains cheaper. The financial case depends on use, although a workload requiring full disconnection may not have public cloud rental as an eligible alternative.

Platforms that deliver a certified true air gap for internal business applications

Procurement needs evidence of disconnected operation for products labeled “private” or “on-premise”. Google Distributed Cloud has hardware and software built to work together that can run disconnected on a customer’s air-gapped network. Google’s appliance documentation also describes a mode in which it stays connected. Gemini models can run on-premises through Vertex AI.

H2O.ai’s case study describes the NIH deploying h2oGPTe inside an air-gapped environment for policy and procurement assistance. The platform supports SLM distillation on private data, while its predictive AI offering includes AutoML and explainability. This fits healthcare and government enterprises that need both predictive and generative AI. The NIH workload also makes the use case concrete, because answering internal policy questions does not require a public-facing application.

Prem AI is built for self-hosted AI and data sovereignty. Its built-in fine-tuning pipeline lets teams upload datasets, train models from more than 30 base architectures, check the results, and deploy to infrastructure they own, all without data leaving the environment. Those architectures include Mistral, LLaMA, Qwen, and Gemma. Fine-tuning is its relevant distinction here. It suits teams that need custom model training within the app generator workflow, rather than a general MLOps platform, with Swiss jurisdiction under the FADP adding a legal layer to its technical controls.

Compliance posture across HIPAA, CMMC, FedRAMP, and AI-specific governance frameworks

A compliance reviewer will ask what data leaves, who can access it, and which controls prove the answer. An air gap addresses the first question through architecture, but it does not settle the whole review. Under HIPAA, keeping PHI inside the organization with no disclosure to the AI vendor removes the need for a BAA with that vendor for such disclosure. A cloud AI service receiving PHI has a different burden, including a signed BAA, processing within its scope, and contractual exclusion of PHI from model training.

Defense requirements need equally precise explanations. CMMC Level 2’s external-connection control calls for connections to be verified and controlled or limited, while allowing them to remain in use. ITAR’s § 120.54 encryption carve-out permits external transmission of unclassified technical data under its encryption conditions. Neither universally mandates an air gap. Classified or already-isolated environments can impose stricter boundaries, but those requirements should not be attributed indiscriminately to every CMMC or ITAR workload.

FedRAMP asks a separate scope question. FedRAMP’s Minimum Assessment Scope rules leave out software that is delivered separately to be installed on agency systems and is not run under a shared responsibility model. However, cloud resources that control or communicate with such software remain in scope if they handle or affect federal data. A provider’s authorization does not automatically cover self-hosted software. SOC 2 Type II also remains relevant to the organization’s own access controls, change management, and audit logging inside the perimeter.

AI-specific governance adds obligations over time alongside those controls. The EU AI Act implementation timeline phases in through 2028, with further transitions to 2030. Keeping the system disconnected is not enough to meet that wider framework’s rules. Nor does it excuse an unexpected outbound route or a component that tries to validate a license remotely. Cryptographic chain-of-custody and the absence of outbound telemetry belong in the compliance evidence because they connect the claimed boundary to the software actually deployed.

Operational model for a small engineering team managing an air-gapped AI deployment

For a small team without staff dedicated to development and operations, most of the ongoing work is in the supply chain. Model updates, OS patches, container refreshes, and vulnerability feeds cannot come through the usual live update channels. A named person should own the perimeter, internal registry, and offline transfer procedure on a defined schedule. Supporting work outside the perimeter, including the connected jumpbox’s CI/CD pipeline, infrastructure automation, and pre-ingress CVE scanning, can be automated.

The connected jumpbox deserves its own controls. It is the highest-risk interface between outside sources and the offline transfer process, even though it is not a live network bridge into the isolated environment. It must have access controls, audit logging, and change management. Every artifact passing through it must be scanned and hashed before transfer. That keeps the operational shortcut of downloading a needed file from becoming an ungoverned exception to the release process.

The internal application should meet the same expectations for reliability and latency as any other production service, with an internal monitoring stack that gets data from metrics endpoints in vLLM or similar frameworks. Isolation is not an excuse for degraded service. The team trades substantial setup complexity for a more stable operating rhythm and costs that do not rise unpredictably with token-metered public API usage.