Running Azure AI Language Containers On-Premises

Core requirement The app must call the Azure AI Language endpoint while confidential documents never leave your own datacenter. The only architecture that satisfies both halves is the hybrid container deployment model: the language analysis itself runs on-premises in the container, and the container reaches Azure only to authenticate and report usage. That means the answer is an ordered three-step set: (1) provision an on-premises Kubernetes cluster that has outbound internet connectivity, (2) pull the Language container image from the Microsoft Container Registry (MCR), and (3) run the container while passing the ApiKey and the Endpoint of your Foundry (Azure AI) resource.

Why each step is required The internet connection on the on-prem cluster is not optional — billing and usage reporting are sent to Azure, so air-gapped clusters cannot run these containers. The image comes from mcr.microsoft.com/azure-ai-*, never Docker Hub; only the container image is hosted publicly, the document content is not. At runtime the two mandatory settings are ApiKey=<your-key> and Endpoint=<your Azure AI resource endpoint>; without both the container will not start its inferencing server on the specified --add-port value.

The adjacent decision The same objective also covers the alternative: a fully managed, cloud-hosted Language resource with no container at all. Choose that when documents are not confidential and you want zero infrastructure. Choose the container path only when data residency or on-premises processing is mandated — and remember that only a subset of Language features (for example, text analytics, custom NER, PII, and language detection) is available as containers; features such as conversational language understanding and question answering are service-only in many regions. Verify the feature's container support before committing.

What would flip the answer If the question removed the "confidential documents remain on-premises" clause, an AKS or any Azure-hosted cluster would be acceptable. If it removed the internet-connectivity requirement, the deployment would fail because usage reporting cannot reach the Azure billing endpoint. Comments on this item reinforce the ordering: provision the on-prem cluster first with internet access, pull from MCR second, then run the container with the API key and endpoint URL third — and avoid AKS, which places processing in the cloud.

Operational consequence of getting it wrong Pulling from Docker Hub yields an unverified or missing image; omitting the endpoint or key causes the container to refuse startup; and hosting the workload on AKS silently violates the confidentiality requirement while appearing technically correct.