Rebuilding the Model Monitor Baseline After a Model Update Causes Data Quality Violations

Monitor model inference.
Answer Correct answer: C — Model Monitor checks data against a stored baseline, so after an update shifts the distribution, a new baseline is created and Model Monitor is pointed at it.

A company has used Amazon SageMaker to deploy a predictive ML model in production. The company is using SageMaker Model Monitor on the model. After a model update, an ML engineer notices data quality issues in the Model Monitor checks. What should the ML engineer do to mitigate the data quality issues that Model Monitor has identified?

  1. Adjust the model's parameters and hyperparameters.
  2. Initiate a manual Model Monitor job that uses the most recent production data.
  3. Create a new baseline from the latest dataset. Update Model Monitor to use the new baseline for evaluations. Correct Answer
  4. Include additional data in the existing training set for the model. Retrain and redeploy the model.

Community Votes

C
67%
D
33%

67% of anonymous learners picked answer C. Votes are pick records left by other test-takers — they are not the verified answer.

Community Insight

Model Monitor data quality constraints are evaluated against a baseline, and when a model update legitimately changes the data distribution, the stale baseline must be recalculated and swapped in so evaluations are meaningful again.

A company runs a deployed SageMaker model under SageMaker Model Monitor, and after a model update the engineer sees data quality issues in the Model Monitor checks. Model Monitor compares incoming production data against a stored baseline that represents expected data distributions, so an update can invalidate that baseline.

Retraining the model with more data or adjusting hyperparameters to fix what is actually a stale-baseline configuration problem. The trigger here is the model update, not a modeling defect, so changing the model does not correct the baseline that Model Monitor evaluates against.

Community Discussion (5 comments)

eesa 👍 1 Selected: C
Amazon SageMaker Model Monitor: Continuously monitors model endpoints to detect issues like: Data quality drift Model quality drift Bias drift Feature attribution drift Baseline: Model Monitor compares incoming data against a baseline dataset that represents "normal" or expected data distributions. If the data distribution changes after a model update, the old baseline may no longer be valid—leading to false positives in data quality issues.
Certified101 👍 1 Selected: C
agree with GiorgioGss - If the problems start appearing "After a model update" then C is the only valid option.
Ell89 👍 2 Selected: D
the model needs to be retrained
Saransundar 👍 1 Selected: C
Model Monitor gives data quality issues --> Create new baseline --> Validate baseline --> Update Model Monitor with new baseline --> Reevaluate data quality --> Investigate and fix root cause (if issues persist) --> Monitor continuously
GiorgioGss 👍 1 Selected: C
If the problems start appearing "After a model update" then C is the only valid option.

Comments & Corrections

No comments yet — spotted an error or have a note? Share it below.

Log in to comment, report an error, or add a note about this question.

Submitted for moderation before publishing. Keep it helpful and respectful.

Expert Analysis

Why the Answer Is Correct

SageMaker Model Monitor evaluates data quality constraints by comparing incoming production data against a baseline dataset that represents the expected normal distribution. When a model update changes the data distribution, that stored baseline no longer reflects reality, so the checks begin reporting data quality issues that reflect the outdated baseline rather than a genuine data problem. The correct remediation is to create a new baseline from the latest dataset, validate it, and point Model Monitor at it for subsequent evaluations. The timeline is the decisive clue: the issues appear after a model update, so the baseline is the stale component. GiorgioGss, Certified101, eesa, and Saransundar all reasoned this way, and Saransundar laid out the full operational sequence of new baseline, validation, update, and re-evaluation.

Why the Other Options Are Wrong

Adjusting model parameters and hyperparameters (A) addresses model behavior, not the reference distribution that Model Monitor checks against, so it leaves the stale baseline in place and the constraints still misfire. Initiating a manual Model Monitor job on the most recent production data (B) only re-runs the checks against the same invalid baseline, so it reproduces the same data quality findings without correcting their cause. Including more data in the training set, retraining, and redeploying (D) treats the signal as a modeling gap, but the scenario attributes the issues to the update changing the distribution rather than to insufficient training data, and it would introduce a new update while the baseline problem remains.

Community Comment Notes

The community voted 67 for C and 33 for D. The dissent, from Ell89, simply stated that the model needs to be retrained, but no comment substantiated that with the after-update timeline. Certified101 and GiorgioGss both observed that if the problems only start appearing after a model update, then C is the only valid option, which captures the core logic of this question.

Official Reference

https://docs.aws.amazon.com/sagemaker/latest/dg/model-monitor-byoc-constraints.html

Related Analysis

An ML engineer trained an ML model on Amazon SageMaker to de

S-Grade

A company needs to create a central catalog for all the comp

A-Grade

A company is building a real-time data processing pipeline f

A-Grade

Practice All MLA-C01 Questions

Access 115 questions with complete answers and detailed explanations.

View Full MLA-C01 Practice Test →

← Back to MLA-C01 Study Guide