TechGlobal.news

Nvidia makes NIM inference microservices generally available on AWS and Azure

Enzo Reno ·

Nvidia said Tuesday it is making its NIM inference microservices generally available on Amazon Web Services and Microsoft Azure, bringing its containerized model endpoints to the two largest public clouds. The company positioned NIM as a standardized way to run Llama, Mistral, and custom models in virtual private clouds with consistent tooling across GPU types and regions.

The move matters for enterprises that want portability without committing to a single provider’s proprietary endpoints. Synergy Research puts AWS at roughly 31 percent cloud infrastructure share, Microsoft Azure at about 25 percent, and Google Cloud near 11 percent as of mid 2025, which puts cross cloud consistency high on procurement agendas.

Nvidia said NIM will support H100 and H200 Tensor Core GPUs on both clouds and L40S where available, with integration for Kubernetes via Helm charts and Nvidia’s GPU Operator. Customers can deploy as managed containers through cloud marketplaces or run on self managed clusters, with observability hooks into Prometheus and OpenTelemetry, according to Nvidia.

The company pointed to MLPerf Inference results as proof of efficiency gains when NIM is paired with Triton Inference Server and TensorRT. In the most recent published MLPerf Inference round, H100 delivered multiple times the throughput of A100 on LLM and vision tasks, and early H200 submissions showed further gains, according to MLCommons. Nvidia said those kernel and graph optimizations are packaged into the NIM images to reduce p95 latency on common LLM prompts.

NIM arrives as hyperscaler managed AI endpoints expand. AWS offers Amazon Bedrock with Titan, Anthropic, and Meta models. Google Cloud’s Vertex AI serves Gemini and third party models. Microsoft provides Azure OpenAI Service alongside model catalogs from Meta and Mistral. Nvidia is pitching NIM as complementary for customers that want to run open weights and proprietary checkpoints under their own VPC controls while retaining GPU portability.

Enterprises evaluating NIM will likely compare it with direct use of Triton on containers they manage. Nvidia said NIM adds curated model images, safety classifiers, retrieval adapters, and prebuilt optimizations, which reduces integration work for teams that do not want to maintain model serving stacks across regions.

Analysts say procurement and governance could drive adoption. Gartner has noted that most large organizations want standardized guardrails and auditability for generative AI by 2026, which favors containerized, policy aware runtimes that span multiple clouds. NIM’s alignment with Kubernetes and cloud marketplaces also fits current enterprise buying patterns.

If NIM gains traction on AWS and Azure, a larger slice of AI inference could shift toward portable runtimes, not provider specific endpoints. That would strengthen the case for hardware abstraction and container based AI stacks across clouds and on premises clusters, and it would intensify competition around GPU availability, schedulers, and cost controls rather than API lock in.