Which SageMaker option mitigates forecast degradation over time?

MLA-C01 By islever Editorial Last updated

Monitor model inference. Analyze model performance.

Short answer

D Use SageMaker Clarify to monitor bias and feature-attribution drift against a baseline and trigger retraining when live demand behavior shifts.

Question

An ecommerce company trains an ML model to forecast demand for near real-time inventory management based on historical customer activity. The company successfully deploys the trained model to a production Amazon SageMaker AI endpoint. However, the company notices that the model’s forecast performance degrades over time. The company needs a long-term and automated solution to mitigate the performance degradation.
Which solution will meet these requirements?
  1. Use Amazon SageMaker Debugger to automatically send alerts when model performance anomalies are detected.
  2. Use AWS X-Ray to monitor the performance of the SageMaker AI endpoint and the incoming requests to inform model re-training.
  3. Use Amazon SageMaker Ground Truth to curate a high-quality dataset. Use the dataset to re-train the model.
  4. Use Amazon SageMaker Clarify to monitor model and feature attribution bias to inform model re-training. Correct Answer

Key Takeaways

This question tests continuous post-deployment drift monitoring for a SageMaker AI endpoint, and the trap is selecting a training-time or request-tracing tool instead of a production model-behavior monitor.

This MLA-C01 question tests how to detect and mitigate post-deployment forecast degradation for a SageMaker AI endpoint. It establishes that SageMaker Clarify monitoring is the best available option here (D), not Debugger, X-Ray, or one-off Ground Truth labeling.

Common mistake: The most common wrong answer is A: SageMaker Debugger sounds like a performance monitor, but it debugs training jobs rather than live endpoint forecast quality.

Expert Analysis

Why the Answer Is Correct

SageMaker Clarify is the only listed service that supports continuous production monitoring of model behavior by comparing live feature attributions and bias metrics with a training baseline. For a demand-forecasting endpoint, a shift in feature attribution is an early signal that customer activity has changed and the model's forecasts are no longer aligned with current demand. Scheduling Clarify monitoring through SageMaker Model Monitor and routing violations to CloudWatch can make this a repeatable, long-term control that informs retraining. Although the option says "bias," Clarify's feature-attribution drift monitoring is the closest available mechanism here for catching post-deployment degradation. The ideal AWS-native answer would be SageMaker Model Monitor for model quality, but it is not offered.

Why the Other Options Are Wrong

SageMaker Debugger (A) is built for training-job diagnostics such as vanishing gradients, overfitting, and resource profiling; it does not continuously evaluate endpoint forecast accuracy. AWS X-Ray (B) traces requests and measures latency and errors, not whether the predicted demand is accurate. SageMaker Ground Truth (C) is a labeling service; curating a high-quality dataset and retraining once does not provide continuous drift detection, and automated labeling does not monitor endpoint performance. None of these options provides the production behavior comparison that Clarify monitoring supplies.

Community Comment Notes

ChiggaBoy correctly spots the "degrades over time" clue as model/data drift, but then chooses Ground Truth (C), which addresses labeling quality rather than continuous monitoring. 9502a22 gives the strongest reasoning for D by explaining that Clarify can monitor data drift and feature-attribution changes and inform retraining. Alabi and sairam2222 support Debugger (A), but their rationale describes training-time anomaly detection rather than live endpoint monitoring. Skri113xX and Chelseajcole illustrate the split vote between C and D, which is why the wording "long-term and automated" is the deciding phrase.

Official Reference

These links open the official vendor documentation in a new tab.

Exam Strategy

When "degrades over time" appears, map it to drift rather than latency. Look for a production monitoring service that compares live data or feature attributions with a baseline; Debugger and X-Ray are for training diagnostics and request tracing, respectively.

Frequently Asked Questions

Why isn't SageMaker Debugger the right tool for endpoint forecast degradation?

Debugger profiles and debugs training jobs; it does not compare live endpoint predictions with ground truth or baseline feature attributions, so it cannot catch this drift.

If Model Monitor is the real service, why is SageMaker Clarify the answer?

Clarify's bias and feature-attribution drift monitors run through SageMaker Model Monitor; among the listed options, D is the only one that continuously monitors production model behavior.

More MLA-C01 FAQ →

Discussion 6 contributions

Compiled from public community threads and reader submissions, lightly edited for clarity. Handles and vote tallies are shown as recorded, and comments reflect the views of individual contributors.

From public community threads 6

  1. Chelseajcole 👍 3 Answered D
    it is D obviously
  2. Alabi 👍 1 Answered A
    A. Use Amazon SageMaker Debugger to automatically send alerts when model performance anomalies are detected. Why A is correct Amazon SageMaker SageMaker Debugger can monitor training and inference behavior to detect anomalies and performance degradation automatically. This provides: continuous monitoring automated alerting long-term operational oversight support for triggering retraining workflows when degradation occurs
  3. 9502a22 👍 1 Answered D
    Clarify can monitor data drift and feature attribution changes in production. It detects when incoming data distributions shift from the training baseline, which directly explains why forecast performance degrades. This monitoring can inform and trigger automated model re-training, making it a long-term, automated solution.
  4. ChiggaBoy 👍 1 Answered C
    The key clue is: “Forecast performance degrades over time.” This is a classic model/data drift scenario. Customer behavior changes, so the historical data the model learned from becomes less representative of current demand.
  5. sairam2222 👍 1 Answered A
    A. Use Amazon SageMaker Debugger to automatically send alerts when model performance anomalies are detected.
Show 1 more comment
  1. Skri113xX 👍 1 Answered C
    I'm voting for C.

Reader comments & corrections

No reader corrections yet — spotted an error or have a note? Share it below.

Log in to comment, report an error, or add a note about this question.

Choosing LightGBM in SageMaker for Fraud Detection with Class Imbalance and Correlated Features

S-grade

Processing Mixed CSV JSON XLSX Parquet Files in One S3 Folder with DataBrew and Outputting Apache Parquet

S-grade

Rebuilding the Model Monitor Baseline After a Model Update Causes Data Quality Violations

S-grade

Orchestrating SageMaker ML Workflows as DAGs in Studio with ML Lineage Tracking for Governance

S-grade

Practice All MLA-C01 Questions

Access 115 questions with complete answers and detailed explanations.

View Full MLA-C01 Practice Test →

← Back to MLA-C01 Study Guide