Deploy WhisperX with post-processing inside one container

PMLE By islever Editorial Last updated

Short answer

C Package the WhisperX model artifacts, post-processing scripts and Hugging Face dependencies into one custom container image on a GPU-enabled Agent Platform endpoint.

Question

You work for a media company. You are developing a solution that requires transcription and diarizing (speaker identification) of live audio streams in near real-time. You are using an optimized WhisperX model built on the PyTorch framework that uses several libraries and weights sourced from the Hugging Face Hub. The deployment must include proprietary Python post-processing logic including speaker diarization and alignment correction that executes immediately after model inference.
You need your solution to be efficient, scalable, handle high throughput, and minimize serving latency. How should you deploy this model and its required logic?
  1. Deploy the WhisperX model artifacts to an Agent Platform endpoint using a pre-built PyTorch serving container configured with a GPU. Host the custom post-processing scripts in a Cloud Run function that receives the endpoint’s prediction output and performs the final diarization.
  2. Use a self-managed Google Kubernetes Engine (GKE) cluster for deployment. Package the model and logic into a container, and configure the cluster’s horizontal pod autoscaler (HPA) to manage throughput.
  3. Package the WhisperX model artifacts, the specific Python post-processing scripts, and the necessary dependencies (including Hugging Face libraries) into a single, custom serving container image. Deploy this custom image to a GPU-enabled Agent Platform endpoint. Correct Answer
  4. Locate the WhisperX model in the Model Garden and use the “1-click deploy” feature to deploy it to an Agent Platform endpoint. Route the output to a Dataflow pipeline that uses a custom DoFn function to handle the post-processing and logging.

Key Takeaways

It tests the boundary between co-located inference and post-processing versus a separate downstream service; the trap is adding a network hop for logic the prompt requires to run immediately after inference.

This PMLE question tests where custom post-processing must live when serving WhisperX in near real-time. The answer is C: bundle model, scripts and dependencies into a single custom container on a GPU Agent Platform endpoint.

Common mistake: Option A is the tempting wrong answer: it looks clean to offload diarization to a Cloud Run function, but that adds latency and its pre-built PyTorch container lacks WhisperX's Hugging Face dependencies.

Expert Analysis

The real boundary: same container vs. separate service The decision this question hinges on is not managed versus self-managed infrastructure — it is whether the post-processing belongs inside the serving container or in a downstream service. A Vertex AI / Agent Platform endpoint accepts a custom container image as its prediction handler, and anything in that image runs in the same process, immediately after inference returns.

Where this scenario falls The prompt is explicit: diarization and alignment correction must execute "immediately after model inference" on near-real-time audio, with minimum latency. That places it firmly on the inside-the-container side of the boundary. Splitting it out — option A's Cloud Run function or option D's Dataflow DoFn — adds a network hop, payload serialization, cold starts and a second scaling domain, all of which work directly against the stated latency and throughput goals.

Why the other side loses Option A fails twice: it also relies on a pre-built PyTorch container, which will not ship WhisperX's specific Hugging Face libraries and weights, so inference may never even run. Option B abandons managed serving for a self-managed GKE cluster with HPA, pushing capacity management onto you without removing any latency cost. Option D additionally assumes a Model Garden one-click publish of WhisperX exists.

Verdict Option C packages artifacts, proprietary scripts and dependencies in one custom image on a GPU endpoint: one process, one autoscaler, no extra hop.

Exam Strategy

When a prompt says custom logic must run immediately after inference, treat any option that moves that logic to a second service as a latency violation and discard it first. Then check whether the serving option can actually supply the model's dependencies: a pre-built framework container cannot carry bespoke Hugging Face weights and libraries. Remaining managed-endpoint options that package everything into one image are almost always the intended answer in serving questions.

Frequently Asked Questions

Why can't the post-processing just run in a Cloud Run function after the endpoint?

Because it adds a network hop, payload serialization and cold-start time between inference and diarization. The prompt requires that logic to run immediately after inference for near-real-time streams, so it belongs in the same process as the model.

Does deploying a custom container mean I manage GPU capacity myself?

No. You own only the container image and its dependencies; the Agent Platform endpoint still manages GPU replicas, health checks and autoscaling, which is what keeps throughput and latency under control.

Would a pre-built PyTorch serving container work for WhisperX?

No. A pre-built image will not include WhisperX's specific Hugging Face libraries, alignment weights and your proprietary diarization code, so inference itself would fail before any post-processing runs.

More PMLE FAQ →

Community Votes

C
66%
D
15%
A
19%

66% of anonymous learners picked answer C. Votes are pick records left by other test-takers — they are not the verified answer.

Discussion 3 contributions

Compiled from public community threads and reader submissions, lightly edited for clarity. Handles and vote tallies are shown as recorded, and comments reflect the views of individual contributors.

From public community threads 3

  1. taylz876 👍 1 Answered C
    Why C is right • The model and the post-processing sit in one container, so there's no extra hop. After WhisperX produces text, the diarization and alignment code runs in the same process, with no network call to another service. That keeps latency as low as possible, which a near-real-time audio stream needs. • Custom dependencies mean a custom container. WhisperX needs specific libraries and weights from the Hugging Face Hub, and the company has proprietary code. A prebuilt container doesn't include these, so you build your own image with everything inside. • A GPU-enabled managed endpoint gives fast inference for audio and autoscales for high throughput, and Google runs the servers. Why the others are wrong • A: The model is on an endpoint, but the post-processing runs in a separate Cloud Run function (now a Cloud Run Function), so every request makes an extra network hop, which adds delay. • B: A self-managed GKE cluster (Kubernetes) means you run and maintain the cluster yourself, which is more operational work than a managed endpoint. • D: One-click deploy covers only the model. The post-processing then runs in a separate Dataflow pipeline, which adds another stage and more delay, and the proprietary logic isn't served with the model.
  2. RajuMVV 👍 1 Answered C
    The correct answer is C. Package the WhisperX model artifacts, the specific Python post-processing scripts, and the necessary dependencies (including Hugging Face libraries) into a single, custom serving container image. Deploy this custom image to a GPU-enabled Agent Platform endpoint.
  3. n2183712847 👍 1 Answered C
    Correct answer: C. Packaging the model, post-processing logic, and all dependencies into a single custom serving container keeps inference and processing together, minimizing network overhead and latency while allowing the GPU-enabled Agent Platform endpoint to scale efficiently.

Reader comments & corrections

No reader corrections yet — spotted an error or have a note? Share it below.

Log in to comment, report an error, or add a note about this question.

Practice All PMLE Questions

Access 65 questions with complete answers and detailed explanations.

View Full PMLE Practice Test →

← Back to PMLE Study Guide