Monitoring Feature Attribution Drift in Vertex AI

PMLE By islever Editorial Last updated

Monitoring, testing, and troubleshooting AI solutions.

Short answer

D Enable request-response logging for the Vertex AI endpoint, set up alerts using Cloud Logging, and review feature attributions in the Google Cloud console.

Question

You developed an ML model using Vertex AI and deployed it to a Vertex AI endpoint. You anticipate that the model will need to be retrained as new data becomes available. You have configured a Vertex AI Model Monitoring Job. You need to monitor the model for feature attribution drift and establish continuous evaluation metrics. What should you do?
  1. Set up alerts using Cloud Logging, and use the Vertex AI console to review feature attributions.
  2. Set up alerts using Cloud Logging, and use Looker Studio to create a dashboard that visualizes feature attribution drift. Review the dashboard periodically.
  3. Enable request-response logging for the Vertex AI endpoint, and set up alerts using Pub/Sub. Create a Cloud Run function to run TensorFlow Data Validation on your dataset.
  4. Enable request-response logging for the Vertex AI endpoint, and set up alerts using Cloud Logging. Review the feature attributions in the Google Cloud console when an alert is received. Correct Answer

Key Takeaways

Feature attribution drift monitoring in Vertex AI requires request-response logging to be enabled so the monitoring job can access live traffic data for calculation.

Learn how to monitor feature attribution drift and continuous evaluation metrics in Vertex AI by enabling request-response logging and using Cloud Logging alerts with the Vertex AI console.

Common mistake: Choosing option C because TensorFlow Data Validation is a powerful tool for drift detection, but it is not the integrated, continuous solution provided by Vertex AI Model Monitoring for operational endpoints.

Expert Analysis

Vertex AI Model Monitoring Capabilities

The question asks how to monitor for feature attribution drift and establish continuous evaluation metrics after deploying a model to a Vertex AI endpoint. The key constraint is that you have already configured a Vertex AI Model Monitoring Job. This service is designed to automatically detect data drift, including feature attribution drift, by analyzing production traffic.

Request-Response Logging Requirement

To calculate feature attributions (such as SHAP values) on live traffic, the monitoring system requires access to both the input features and the model's predictions. Vertex AI Model Monitoring achieves this by sampling traffic and storing it in BigQuery. This process relies on request-response logging being enabled for the endpoint. Without this logging, the monitoring job cannot access the necessary data to compute attributions or evaluate performance against ground truth labels (if provided). Therefore, enabling request-response logging is a prerequisite for the monitoring job to function correctly for these specific metrics.

Alerting and Review Mechanism

Once request-response logging is enabled, the Model Monitoring Job runs continuously. It generates reports and triggers alerts when drift thresholds are exceeded. These alerts are sent to Cloud Logging (and optionally Cloud Monitoring/Pub/Sub for custom actions). To review the specific details of feature attribution drift, users typically use the Vertex AI Console, which visualizes the monitoring reports and attribution scores derived from the logged data. While external tools like Looker Studio can visualize this data, the native console provides the direct interface for reviewing drift reports generated by the monitoring job. Alerts via Cloud Logging ensure timely notification.

Why Other Options Are Incorrect

Option A suggests using Cloud Logging alerts but reviewing attributions without explicitly enabling request-response logging, which is technically insufficient for the monitoring job to compute attributions on live traffic. Option B involves creating a custom Looker Studio dashboard, which is unnecessary complexity when the Vertex AI Console provides built-in visualization for monitoring reports. Option C proposes using Pub/Sub and a Cloud Run function with TensorFlow Data Validation (TFDV). While TFDV can detect drift, it is not the standard mechanism for continuous operational monitoring within Vertex AI; the Vertex AI Model Monitoring service handles this natively. Furthermore, setting up a custom pipeline for every retraining cycle contradicts the goal of leveraging the managed monitoring service.

Conclusion

The correct approach combines enabling the underlying data collection mechanism (request-response logging) with the native alerting and review workflows provided by Vertex AI. This ensures the monitoring job has the data it needs to calculate feature attributions and that administrators are alerted via Cloud Logging to review the results in the console.

Official Reference

https://cloud.google.com/vertex-ai/docs/model-monitoring/overview

This link opens the official vendor documentation in a new tab.

Exam Strategy

When dealing with Vertex AI Model Monitoring questions, always verify if request-response logging is mentioned as a requirement. If the question asks about monitoring data drift or feature attributions on live traffic, enabling request-response logging is almost always the foundational step. Focus on the integration between the monitoring job, the logging mechanism, and the native console for reporting rather than building custom pipelines unless explicitly asked.

Frequently Asked Questions

Why is request-response logging required for feature attribution drift?

Feature attribution algorithms like SHAP require both input features and model predictions to calculate importance scores. Vertex AI Model Monitoring samples this data from request-response logs to perform these calculations on production traffic.

Can I use Looker Studio instead of the Vertex AI Console?

Yes, you can export monitoring reports to BigQuery and visualize them in Looker Studio. However, the Vertex AI Console provides a native, pre-built interface for reviewing drift and attribution reports, making it the simpler and more direct choice for most scenarios.

What happens if I don't enable request-response logging?

Without request-response logging, the Vertex AI Model Monitoring Job cannot access the live traffic data needed to compute feature attributions or evaluate performance against ground truth. Consequently, drift detection for these metrics will fail or provide incomplete data.

More PMLE FAQ →

Community Votes

D
57%
A
43%

57% of anonymous learners picked answer D. Votes are pick records left by other test-takers — they are not the verified answer.

Discussion 7 contributions

Compiled from public community threads and reader submissions, lightly edited for clarity. Handles and vote tallies are shown as recorded, and comments reflect the views of individual contributors.

From public community threads 7

  1. qaz09 👍 4 Answered A
    not C, D -> If an endpoint has Model Monitoring enabled, you can't enable request-response logging for the same endpoint. https://cloud.google.com/vertex-ai/docs/predictions/online-prediction-logging#model-monitoring A is simpler solution than B, I don't think there is a need to create custom Looker Studio dashboards, since you can use Vertex AI console.
  2. hit_cloudie 👍 4 Answered D
    This leverages Vertex AI Model Monitoring with request-response logging, Cloud Logging alerts, and built-in feature attribution drift review.
  3. winston9 👍 2 Answered D
    the monitoring service and manual request-response logging both utilize the same underlying BigQuery mechanism; but still, D remains the best answer because of how Model Monitoring v1 actually operates on the backend to satisfy the requirement of "monitoring feature attribution drift
  4. taylz876 👍 1 Answered D
    Why D is right • Feature attribution drift needs the live traffic data. To calculate attributions on production traffic, the endpoint has to save the incoming requests and returned predictions. Enabling request-response logging does that, and it gives the raw data to measure drift and to build continuous evaluation metrics. • Alerts through Cloud Logging: when the monitoring job sees the attributions drifting from the baseline, you're alerted without having to check by hand. • Review in the console: you open the Agent Platform console to see which features' attributions have shifted. The visualization is built in. Why the others are wrong • A: It sets up alerts and reviews attributions, but skips enabling request-response logging, so the endpoint doesn't save the payloads that the attributions and evaluation metrics need. • B: A custom Looker Studio (now Data Studio) dashboard reviewed periodically is extra work, and the console already shows feature attribution drift. • C: Pub/Sub plus a Cloud Run function running TensorFlow Data Validation checks raw data and schema drift. It doesn't track feature attribution drift, which needs the model's predictions.
  5. Johnb9682 👍 1 Answered D
    1. Enable Request-Response Logging on the Endpoint: 2. Set Up Alerts Using Cloud Logging / Cloud Monitoring 3. Review Feature Attributions in the Google Cloud Console
Show 2 more comments
  1. dija123 👍 1 Answered A
    Totally agree with A.
  2. el_vampiro 👍 1 Answered A
    qaz01 is right

Reader comments & corrections

No reader corrections yet — spotted an error or have a note? Share it below.

Log in to comment, report an error, or add a note about this question.

Monitoring ML Model Performance with Precision and Recall Sampling

S-grade

Optimizing ML Model Recall for Compliance Risk

S-grade

How to Minimize Development Time for Weekly TensorFlow Retraining in Vertex AI?

S-grade

Building CLV Models with BigQuery ML in Vertex AI Workbench

S-grade

Practice All PMLE Questions

Access 65 questions with complete answers and detailed explanations.

View Full PMLE Practice Test →

← Back to PMLE Study Guide