Enabling Request-Response Logging for Model Evaluation

PMLE By islever Editorial Last updated

Short answer

C Enable request-response logging to BigQuery with a sampling rate of 1.0 on the Agent Platform endpoint to capture all predictions for exact metric calculation.

Question

You have deployed a custom classification model to an Agent Platform endpoint to predict the likelihood of customer churn in real-time. You need to collect the data to calculate the model’s exact precision and recall by comparing production predictions with ground truth data in BigQuery. You want to minimize development effort. What should you do?
  1. Modify the prediction container to asynchronously write the prediction result to BigQuery after every request.
  2. Schedule a daily Agent Platform batch prediction job that uses the same model resource as the deployed endpoint and stores predictions in BigQuery.
  3. Update the model deployment parameters to enable request-response logging to BigQuery. Set the sampling rate to 1.0. Correct Answer
  4. Create a model evaluation job by providing the model resource and the BigQuery ground truth table to calculate precision and recall.

Key Takeaways

The question tests understanding of Agent Platform's built-in request-response logging feature versus custom coding or batch approaches for capturing online serving data.

Learn how to enable request-response logging on Google Cloud Agent Platform endpoints to capture production predictions in BigQuery for precise model evaluation metrics like precision and recall.

Common mistake: Many candidates choose A (modifying the container) thinking they need custom code to log data, but this ignores the platform's native logging capability that requires zero development effort.

Expert Analysis

The core challenge is capturing real-time production predictions in BigQuery to calculate exact precision and recall with minimal development effort. The key constraint is minimizing code changes while ensuring every prediction is logged for accurate metric calculation.

Option C is the correct approach because Agent Platform endpoints support request-response logging directly to BigQuery as a deployment parameter. Setting the sampling rate to 1.0 ensures that every single prediction request and response is captured, which is necessary for calculating exact precision and recall by joining with ground truth data. This requires no custom application code or additional infrastructure setup beyond configuration.

Option A is incorrect because modifying the prediction container adds significant development effort and operational complexity, violating the minimize effort requirement. Option B is wrong because batch prediction does not capture real-time online serving traffic, so it would miss the actual predictions made during customer interactions. Option D describes model evaluation, which calculates metrics but does not address the fundamental problem of getting the prediction data into BigQuery in the first place; without logging the predictions, there is nothing to evaluate against ground truth.

Exam Strategy

When questions ask for minimal development effort to capture production data, look for platform-native features first. Agent Platform provides request-response logging as a configuration option, eliminating the need for custom code modifications or complex pipeline setups.

Frequently Asked Questions

Why is sampling rate 1.0 important for exact precision and recall?

A sampling rate of 1.0 ensures every single prediction request and response is logged to BigQuery. Partial sampling would result in incomplete data, making it impossible to calculate exact precision and recall metrics accurately.

Can I use batch predictions instead of real-time logging?

No, batch predictions process historical data offline and do not capture the actual real-time predictions made during online serving. To compare predictions with ground truth, you need the actual predictions served to customers.

More PMLE FAQ →

Community Votes

C
100%

100% of anonymous learners picked answer C. Votes are pick records left by other test-takers — they are not the verified answer.

Discussion 2 contributions

Compiled from public community threads and reader submissions, lightly edited for clarity. Handles and vote tallies are shown as recorded, and comments reflect the views of individual contributors.

From public community threads 2

  1. n2183712847 👍 1 Answered C
    Correct answer: C. Enabling request-response logging with a 1.0 sampling rate captures every production request and prediction in BigQuery, providing the prediction data needed to join with ground truth and calculate exact precision and recall with minimal development effort; A adds custom application code, B creates unnecessary batch predictions instead of capturing real-time traffic, and D cannot calculate production precision/recall unless the production predictions have first been collected.
  2. taylz876 👍 1 Answered C
    Similar to Q318 Why C is right • You need production predictions in BigQuery to compare with ground truth. Request-response logging saves each request and the model's response to a BigQuery table, which you can join with the ground truth table. • "Exact" precision and recall need every prediction, so a sampling rate of 1.0 logs all of them. • Minimal effort: it's a deployment setting, with no code changes.

Reader comments & corrections

No reader corrections yet — spotted an error or have a note? Share it below.

Log in to comment, report an error, or add a note about this question.

Practice All PMLE Questions

Access 65 questions with complete answers and detailed explanations.

View Full PMLE Practice Test →

← Back to PMLE Study Guide