Low-latency streaming predictions with automatic model reload

PMLE By islever Editorial Last updated

Short answer

D Use the RunInference API with WatchFilePattern in a Dataflow job so predictions run in-process for low latency and the model reloads automatically from Cloud Storage when updated.

Question

Your company manages an ecommerce website. You developed an ML model that recommends additional products to users in near real time based on items currently in the user’s cart. The workflow will include the following processes:
  1. The website will send a Pub/Sub message with the relevant data and then receive a message with the prediction from Pub/Sub
  2. Predictions will be stored in BigQuery
  3. The model will be stored in a Cloud Storage bucket and will be updated frequently
You want to minimize prediction latency and the effort required to update the model. How should you reconfigure the architecture?
  1. Write a Cloud Function that loads the model into memory for prediction. Configure the function to be triggered when messages are sent to Pub/Sub.
  2. Create a pipeline in Vertex AI Pipelines that performs preprocessing, prediction, and postprocessing. Configure the pipeline to be triggered by a Cloud Function when messages are sent to Pub/Sub.
  3. Expose the model as a Vertex AI endpoint. Write a custom DoFn in a Dataflow job that calls the endpoint for prediction.
  4. Use the RunInference API with WatchFilePattern in a Dataflow job that wraps around the model and serves predictions. Correct Answer

Key Takeaways

The question's dual constraint — low latency AND minimal model-update effort — is the test; the trap is picking an option strong on only one of the two.

For near-real-time recommendations from Pub/Sub, RunInference inside a Dataflow job with WatchFilePattern minimizes prediction latency and requires no redeployment when the Cloud Storage model changes, making option D correct.

Common mistake: Option A, because loading the model into Cloud Function memory seems to cut latency, but it forces a redeployment every time the frequently updated model changes and risks cold starts.

Expert Analysis

Two goals, one conjunction The question asks you to satisfy both goals simultaneously: minimize prediction latency and minimize the effort required to update a model that "will be updated frequently" from a Cloud Storage bucket. Any option that wins on one criterion but loses on the other must be eliminated.

Ruling out option A A Cloud Function triggered by Pub/Sub does load the model into memory, which helps latency, but the function's source code (or its attached package) must be redeployed every time the model artifact changes. For a model that is updated frequently, that is exactly the effort the question wants to minimize. It also risks cold-start latency and per-invocation concurrency limits for near-real-time ecommerce traffic.

Ruling out option B A Vertex AI Pipeline is a batch/orchestration construct. Starting a full pipeline run (preprocessing, prediction, postprocessing) per Pub/Sub message adds queueing and startup overhead, and it still requires re-running the pipeline with a new model version — worst on both counts.

Ruling out option C A Vertex AI endpoint gives excellent online latency, but "exposing the model as an endpoint" means you must redeploy (or traffic-split) the endpoint each time the Cloud Storage model is updated. That is precisely the update effort being minimized.

Why option D is correct RunInference executes the model inside the Dataflow pipeline, so predictions happen in-process with no extra network hop to a separate prediction service — low latency. Critically, WatchFilePattern watches the Cloud Storage location and reloads the model automatically when a new artifact appears, so updates require no redeployment at all. The Pub/Sub-in / BigQuery-out flow already described matches Dataflow's streaming model, making D the only choice that satisfies both qualifiers at once.

Official Reference

Exam Strategy

When a question says minimize X and Y, treat it as a two-column elimination: score each option against both criteria and discard any that fails one. Here, options A and C fail update effort, B fails latency, leaving D. Also notice the workflow already specifies Pub/Sub in and BigQuery out, which points to a Dataflow streaming job rather than a serverless function or a serving endpoint.

Frequently Asked Questions

Why not call a Vertex AI endpoint from a Dataflow DoFn for low latency?

An endpoint adds a network round trip per prediction and must be redeployed whenever the Cloud Storage model changes, failing the question's update-effort constraint.

What does WatchFilePattern actually do in RunInference?

It monitors the Cloud Storage path of the model artifact and automatically reloads the model inside the running pipeline when a new version is written, with no redeployment.

Is a Cloud Function wrong for near-real-time predictions here?

It can serve predictions, but frequent model changes require redeploying the function and cold starts add latency, so it does not satisfy both stated goals.

More PMLE FAQ →

Community Votes

D
50%
A
30%
C
20%

50% of anonymous learners picked answer D. Votes are pick records left by other test-takers — they are not the verified answer.

Discussion 11 contributions

Compiled from public community threads and reader submissions, lightly edited for clarity. Handles and vote tallies are shown as recorded, and comments reflect the views of individual contributors.

From public community threads 11

  1. guilhermebutzke 👍 7 Answered D
    My answer: D This Google Documentation explains “Instead of deploying the model to an endpoint, you can use the RunInference API to serve machine learning models in your Apache Beam pipeline. This approach has several advantages, including flexibility and portability.” https://cloud.google.com/blog/products/ai-machine-learning/streaming-prediction-with-dataflow-and-vertex This documentation uses RunInference and WatchFilePattern to “to automatically update the ML model without stopping the Apache Beam”. https://cloud.google.com/dataflow/docs/notebooks/automatic_model_refresh So, thinking in “minimize prediction latency”, its suggested use RunInfenrece, while “effort required to update the model” the WatchFilePattern is the best approach. I think D is the best option
  2. 4d742d7 👍 4 Answered C
    Vertex AI endpoints are built for online prediction and automatically serve updated models when deployed. You don’t need to reload the model in memory manually. Dataflow is scalable, stream-friendly, and integrates well with Pub/Sub for input and BigQuery for output. Writing a custom DoFn in Dataflow gives you flexibility to call the endpoint using the prediction API and process results efficiently.
  3. ddogg 👍 4 Answered D
    Automatic Model Updates: WatchFilePattern automatically detects model changes in Cloud Storage, leading to seamless updates without managing endpoint deployments.
  4. lunalongo 👍 3 Answered A
    A is the best option because: - Minimizes Latency: Loading the model into the Cloud Function's memory eliminates the overhead of loading the model from storage for each prediction request. This significantly reduces latency, crucial for near real-time recommendations. The function is triggered directly by Pub/Sub messages, further streamlining the process. - Simplified Model Updates: Updating the model involves simply deploying a new version of the Cloud Function with the updated model. This is a much simpler process than managing pipelines or endpoints. D is the most voted so far, but... The complexity of managing the Dataflow pipeline and the potential latency introduced by the pipeline outweigh the benefits of automatic model updates using WatchFilePattern in this context. Therefore, option A (Cloud Function) remains the most efficient solution.
  5. pikachu007 👍 2 Answered A
    Low Latency: Serverless Execution: Cloud Functions start up almost instantly, reducing prediction latency compared to alternatives that require longer setup or deployment times. In-Memory Model: Loading the model into memory eliminates disk I/O overhead, further contributing to rapid predictions.
Show 6 more comments
  1. CHARLIE2108 👍 2
    Cloud Functions offer low latency but it might not scale well.
  2. taylz876 👍 1 Answered D
    Why D is right The flow is already Pub/Sub in and Pub/Sub out, and you want the lowest prediction latency and the least effort to update the model. • Low latency: RunInference runs the model inside the Dataflow pipeline itself, so there is no network call to a separate prediction service. • Easy model updates: WatchFilePattern watches the Cloud Storage location where the model is saved. When a new model file appears, the pipeline loads it automatically, with no restart and no redeployment. • It fits the architecture: Dataflow reads from Pub/Sub and can write the predictions back to Pub/Sub and to BigQuery. Why the others are wrong • A: A Cloud Function that loads the model into memory is slow on cold starts and needs redeploying whenever the model changes. It is not built for frequent updates. • B: A Vertex AI Pipelines run is a heavy batch workflow, so it adds a lot of latency for every request in a near-real-time use case. • C: Calling a Vertex AI endpoint from a custom DoFn adds a network hop to every prediction, which increases latency. You would also have to redeploy the model each time it is updated.
  3. phani49 👍 1 Answered D
    Exposing the model as a Vertex AI endpoint and using Dataflow with a custom DoFn provides the optimal solution for real-time predictions with minimal latency. https://cloud.google.com/blog/products/ai-machine-learning/streaming-prediction-with-dataflow-and-vertex
  4. PhilipKoku 👍 1 Answered C
    C) Expose the model as Vertex AI End Point
  5. pinimichele01 👍 1 Answered D
    agree with guilhermebutzke
  6. Yan_X 👍 1 Answered A
    A for me.

Reader comments & corrections

No reader corrections yet — spotted an error or have a note? Share it below.

Log in to comment, report an error, or add a note about this question.

Practice All PMLE Questions

Access 65 questions with complete answers and detailed explanations.

View Full PMLE Practice Test →

← Back to PMLE Study Guide