How to Optimize Real-Time ML Prediction Latency on GCP?

You are an ML engineer at a retail company. You have built a model that predicts a coupon to offer an ecommerce customer at checkout based on the items in their cart. When a customer goes to checkout, your serving pipeline, which is hosted on Google Cloud, joins the customer's existing cart with a row in a BigQuery table that contains the customers' historic purchase behavior and uses that as the model's input. The web team is reporting that your model is returning predictions too slowly to load the coupon offer with the rest of the web page. How should you speed up your model's predictions?

Attach an NVIDIA P100 GPU to your deployed model’s instance.
Use a low latency database for the customers’ historic purchase behavior. Source Reference Answer
Deploy your model to more instances behind a load balancer to distribute traffic.
Create a materialized view in BigQuery with the necessary data for predictions.

Community Votes

57%

43%

57% of anonymous learners picked answer B. Votes are pick records left by other test-takers — they are not the verified answer.

Community Insight

Tests architectural understanding of serving bottlenecks, with the common trap being the assumption that BigQuery materialized views can efficiently handle dynamic, per-request joins for real-time inference.

This scenario evaluates strategies for reducing inference latency when combining transactional cart data with historical customer analytics. The community consensus strongly advocates migrating analytic data stores like BigQuery to low-latency NoSQL databases for online prediction serving.

Option D was frequently selected because materialized views accelerate repeated analytical queries, but they are ineffective here since they cannot precompute joins involving highly dynamic, session-specific cart items at prediction time.

Community Discussion (16 comments)

desertlotus1211 👍 1 Selected: B

You want to use Bigtable, Firestore, or Memorystore, or maybe ReDIS

NamitSehgal 👍 1 Selected: D

Materialized views directly address this bottleneck by pre-computing the join.

vini123 👍 1 Selected: B

If the primary issue is real-time access and speed, Option B is probably the better choice, as low-latency databases are built specifically for that purpose.

DaleR 👍 2 Selected: D

Keep everything in BigQuery. Migrating to a fast database is more complex and can potentially introduce challenges.

f084277 👍 2 Selected: B

Unclear how an MV would help retrieve a single row any faster. Something like BigTable (a low latency database) would be much faster.

inc_dev_ml_001 👍 2 Selected: B

It says that you have to join the cart data, so you can't use the materialized view because it means that you should materialize the view every time a new cart shows up. So use a low latency DB it's the only way

inc_dev_ml_001 👍 2 Selected: B

In my opinion the materialized view could be the best way but it says that the cart data have to join with historic behaviour so it's impossibile to have all the needed data for the prediction in the materialized view because cart data are not in the database.

SausageMuffins 👍 1 Selected: D

Both B and D in theory does reduce latency but B implies that we might need to migrate the database to another low latency database. This migration and setup might incur additional costs and effort. In contrast, creating a materialized view seems much more straight forward since there is already a preexisting big query table mentioned in the question.

Ria_1989 👍 1

Coupon to offer an ecommerce customer at checkout based on the items in their cart not the customer historic behaviour. That's creating confusion while choosing B.

fitri001 👍 2 Selected: D

Reduced Join Cost: Joining the customer's cart with their purchase history in BigQuery during each prediction can be slow. A materialized view pre-computes and stores the join results, eliminating the need for repetitive joins and significantly reducing latency. Targeted Data Access: Materialized views allow you to specify the exact columns needed for prediction, minimizing data transferred between BigQuery and your serving pipeline.

gscharly 👍 4 Selected: B

https://cloud.google.com/architecture/minimizing-predictive-serving-latency-in-machine-learning#online_real-time_prediction "Analytical data stores such as BigQuery are not engineered for low-latency singleton read operations, where the result is a single row with many columns."

guilhermebutzke 👍 3 Selected: B

I changed my mind. B: Im read a lot this page https://cloud.google.com/architecture/minimizing-predictive-serving-latency-in-machine-learning#online_real-time_prediction If the web team is reporting that the model is returning predictions too slowly to load the coupon offer with the rest of the web page, it suggests that the bottleneck might indeed be in the inference process rather than in data retrieval or processing. Given that the model is deployed on Google Cloud, choosing a low-latency database makes it suitable for scenarios where quick access to data is crucial, such as real-time predictions for web applications. Option D: While pre-aggregating data in BigQuery can improve query speed, it might not be as efficient as a low-latency database for frequently accessed data like customer purchase history.

guilhermebutzke 👍 1 Selected: D

Firstly, I believe the correct choice should be B. This is supported by a comprehensive Google page discussing methods to minimize real-time prediction latency. In this resource, they don't mention using a BigQuery view but instead suggest precomputing and lookup approaches to minimize prediction time. https://cloud.google.com/architecture/minimizing-predictive-serving-latency-in-machine-learning#online_real-time_prediction However, I will stick with option D because it's not clear whether option B suggests changing the entire database or just utilizing it as a preliminary step for online prediction.

sonicclasps 👍 2 Selected: D

Queries that use materialized views are generally faster and consume fewer resources than queries that retrieve the same data only from the base tables. Materialized views can significantly improve the performance of workloads that have the characteristic of common and repeated queries.

ddogg 👍 3 Selected: D

D. Create a materialized view in BigQuery with the necessary data for predictions. Here's why: Current bottleneck: Joining the cart data with the BigQuery table containing historic purchases likely creates the latency bottleneck. Fetching data from BigQuery on every prediction request can be slow. Materialized view: A materialized view pre-computes and stores the join between the cart data and the relevant historic purchase information in BigQuery. This eliminates the need for real-time joins during prediction, significantly reducing latency. Faster access: The pre-computed data in the materialized view is readily available within BigQuery, ensuring faster access for your serving pipeline when predicting the coupon offer. Lower cost: Compared to additional instances or GPU resources, a materialized view can be a more cost-effective solution, especially if prediction requests are frequent.

kalle_balle 👍 1 Selected: B

Option B seems most sensible.

Comments & Corrections

No comments yet — spotted an error or have a note? Share it below.

Expert Analysis

BigQuery is fundamentally an OLAP system designed for analytical workloads, not low-latency singleton reads required during real-time inference. To minimize prediction latency, you must offload historical feature retrieval to a purpose-built online database like Cloud Bigtable, Firestore, or Memorystore. These services are optimized for fast key-value lookups and consistently deliver sub-millisecond response times under heavy concurrent traffic.

Why the Other Options Are Wrong

Attaching a GPU accelerates matrix multiplications during model training or inference but does nothing to resolve I/O bottlenecks caused by external data fetches. Deploying additional instances behind a load balancer increases throughput capacity but leaves the underlying data retrieval delay completely unchanged. Creating a materialized view in BigQuery precomputes static joins for reporting dashboards, but it cannot dynamically incorporate unpredictable, session-specific cart items during live checkout requests.

Community Comment Notes

Comment [1] directly cites Google’s official architecture documentation, which explicitly warns against using analytical data stores for real-time prediction serving. Comments [6] and [7] correctly identified that materialized views fail here because they cannot account for highly volatile cart data that changes per transaction. Comment [10] reinforced the optimal solution by recommending specialized low-latency stores like Bigtable or Memorystore for feature lookup pipelines.

Official Reference

https://cloud.google.com/architecture/minimizing-predictive-serving-latency-in-machine-learning#online_real-time_prediction

Exam Strategy

When optimizing real-time ML serving, always separate your analytics pipeline from your feature serving layer using purpose-built low-latency databases. Rely on precomputed features or offline joins only when the underlying data remains static between requests, and avoid routing live checkout traffic through analytical databases regardless of cluster scaling.

Related Analysis

Practice All PMLE Questions

Access 65 questions with complete answers and detailed explanations.

View Full PMLE Practice Test →

← Back to PMLE Study Guide