Nvidia makes NIM inference microservices generally available on Red Hat OpenShift for private AI deployments

Nvidia said Tuesday that its NIM inference microservices are now generally available on Red Hat OpenShift, giving enterprises a supported path to run generative AI behind their firewalls. The release targets regulated industries that want standardized model serving on Kubernetes, with support for Meta Llama 3.1, Mistral models, Nvidia Nemotron, and speech models via Riva on H100, H200, and L40S GPUs.

Nvidia is packaging NIM with its AI Enterprise subscription, which the company lists at 4,500 dollars per GPU per year for perpetual subscriptions, or one dollar per GPU hour for cloud consumption. The move matters as enterprises weigh private deployments to meet data residency and logging obligations under the EU AI Act, which enters phased enforcement in 2026 according to the European Commission.

The OpenShift support arrives with validated designs from Dell Technologies, Hewlett Packard Enterprise, and Lenovo for eight GPU and sixteen GPU servers, along with documented integrations for vector stores from Milvus, Pinecone, Redis, and PostgreSQL with pgvector. Red Hat said NIM images can be deployed through OpenShift AI, its model lifecycle toolkit that runs on OpenShift, and monitored with Prometheus and Grafana, which simplifies operations for platform teams that already standardize on OpenShift.

Nvidia positioned NIM as a complement to Triton Inference Server and TensorRT LLM, exposing REST and gRPC endpoints with token streaming, batching, and dynamic quantization options. The company published reference guides for Llama 3.1 70B and Mixtral 8x7B that cover prompt management, safety filters via Llama Guard, and tool use routing, which are common enterprise requirements. Nvidia also highlighted connectors for data governance through Immuta and Privacera for customers that need audit trails.

Pricing anchors the comparison against cloud managed APIs. OpenAI lists GPT 4o mini at 15 cents per 1,000 input tokens and 60 cents per 1,000 output tokens, while Anthropic lists Claude 3.5 Sonnet at 3 dollars per 1,000 input tokens and 15 dollars per 1,000 output tokens on its pricing pages. Nvidia’s approach shifts cost to infrastructure and software subscription, which appeals to customers that already negotiate server contracts and want predictable spend tied to GPUs rather than tokens.

Availability starts today for Red Hat OpenShift customers with active Nvidia AI Enterprise subscriptions, with support through Nvidia and Red Hat. Dell, HPE, and Lenovo plan to publish updated validated designs for H200 and L40S based stacks this quarter, with Cisco citing forthcoming UCS reference architectures for OpenShift AI.

For the broader technology landscape, the announcement underscores a bifurcation between public cloud managed AI services from AWS, Microsoft, and Google, and private clusters built on Kubernetes with commercial support. As model quality converges across open and proprietary options, procurement is shifting to platform standardization, operations tooling, and compliance readiness, which puts vendors that can package inference, observability, and policy controls into existing enterprise stacks at an advantage.