Continuous Retraining Strategy for Evolving User Preferences

PDE By islever Editorial Last updated

Preparing data for AI and ML. Organizing workloads based on business requirements.

Short answer

B Continuously retrain the model on a combination of existing data and the new data to balance adaptation to changing preferences with retention of historical context.

Question

You are building a model to make clothing recommendations. You know a user's fashion preference is likely to change over time, so you build a data pipeline to
stream new data back to the model as it becomes available. How should you use this data to train the model?
  1. Continuously retrain the model on just the new data.
  2. Continuously retrain the model on a combination of existing data and the new data. Correct Answer
  3. Train on the existing data while using the new data as your test set.
  4. Train on the new data while using the existing data as your test set.

Key Takeaways

The main trap is confusing retraining with testing; you must never use new streaming data as a test set for training, nor should you discard historical data which prevents catastrophic forgetting.

Learn why combining existing and new data is the correct approach for continuously retraining recommendation models to handle changing user preferences without losing historical insights.

Common mistake: Option A is the most plausible wrong answer because it assumes 'continuously' means using only the latest data. However, this ignores the need for historical context, leading to a model that forgets past patterns and overfits to recent noise.

Expert Analysis

The core challenge in this scenario is managing concept drift, where user fashion preferences evolve over time. When a pipeline streams new data back to the model, the goal is to maintain predictive accuracy without forgetting established patterns. Option B, retraining on a combination of existing and new data, is the standard best practice for continuous learning in recommendation systems. This approach, often called 'incremental' or 'periodic full retraining with historical context,' ensures the model adapts to recent trends while retaining the statistical robustness provided by historical data.

Option A is incorrect because training solely on new data leads to 'catastrophic forgetting,' where the model loses knowledge of long-term user behaviors, resulting in poor generalization. Options C and D are fundamentally flawed because they misuse the test set as training data. In machine learning operations (MLOps), the test set must remain unseen during training to provide an unbiased evaluation of model performance. Using new data as a test set (Option D) or existing data as a test set while ignoring new data for training (Option C) violates the principle of data integrity and prevents the model from actually learning from the incoming stream.

This question tests your understanding of MLOps lifecycle management, specifically the balance between adaptability and stability. The adjacent decision involves monitoring metrics to trigger retraining; if performance degrades, you combine historical and new data. Getting this wrong would lead to models that either stagnate or become unstable. The operational consequence of using only new data is a volatile model sensitive to short-term noise, while mixing training/test sets causes data leakage and invalid evaluations.

Exam Strategy

When dealing with 'continuous' or 'streaming' data scenarios in ML questions, always look for the option that preserves historical context. Avoid answers that suggest discarding old data (leading to forgetting) or using new data for validation during the training phase (data leakage). Prioritize solutions that mention 'combination', 'hybrid', or 'periodic updates' involving both datasets.

Frequently Asked Questions

Why is training on just new data bad?

Training only on new data causes 'catastrophic forgetting,' where the model loses its ability to generalize based on long-term historical patterns, making recommendations unstable.

Can I use new data as a test set?

No. Test sets must be held out from training to evaluate performance objectively. Using new streaming data as a test set while training elsewhere breaks the evaluation integrity.

More PDE FAQ →

Community Votes

B
94%
C
6%

94% of anonymous learners picked answer B. Votes are pick records left by other test-takers — they are not the verified answer.

Discussion 54 contributions

Compiled from public community threads and reader submissions, lightly edited for clarity. Handles and vote tallies are shown as recorded, and comments reflect the views of individual contributors.

From public community threads 30 of 54

  1. serg3d 👍 39
    I think it should be B because we have to use a combination of old and new test data as well as training data
  2. jagadamba 👍 12
    B, as we need to train the data with new data, so that it will keep learning, and as well as used for test
  3. samdhimal 👍 7 Answered B
    B. Continuously retrain the model on a combination of existing data and the new data. This approach will help to ensure that the model remains up-to-date with the latest fashion preferences of the users, while also leveraging the historical data to provide context and improve the accuracy of the recommendations. Retraining the model on a combination of existing and new data will help to prevent the model from being overly influenced by the new data and losing its ability to generalize to users with different preferences. Option A is not recommended because retraining the model on just new data will cause the model to lose the information it has learned from the historical data. Option C and D are not recommended because they are using the new data as test set and this approach will lead to a model that is overfitting and not generalize well to new users.
  4. kino2020 👍 7
    Quoted from that article because it says to build the PiPeLine. Producing evaluation metric values using the trained model on a test dataset to assess the model's predictive quality. Comparing the evaluation metric values produced by your newly trained model to the current model, for example, production model, baseline model, or other business-requirement models. You make sure that the new model produces better performance than the current model before promoting it to production. https://cloud.google.com/solutions/machine-learning/mlops-continuous-delivery-and-automation-pipelines-in-machine-learning Therefore, B is the correct answer.
  5. GregDT 👍 6
    This article seems to support answer B. https://medium.com/codait/keeping-your-machine-learning-models-up-to-date-f1ead546591b. Here's a quote from this article: "If you see the accuracy of your model degrading over time, use the new data, or a combination of the new data and old training data to build and deploy a new model."
Show 25 more comments
  1. Archy 👍 6
    B, continuous training is needed to keep your model upto date
  2. dambilwa 👍 5
    Yes - The training set should be shuffled well to represent data across all scenarios
  3. MaxNRG 👍 4
    I would go with B (but A is also possible If the market is fast changing) https://datascience.stackexchange.com/questions/12761/should-a-model-be-re-trained-if-new-observations-are-available: Suppose that your model attempts to predict customers' behavior, e.g. how likely is a customer to purchase your product given an offer tailored for him. Clearly, the market changes over time, customers' preferences change, and your competitors adjust. You should adjust as well, so you need to retrain periodically. In such a case I would recommend to add new data, but also omit old data that is not relevant anymore. If the market is fast changing, you should even consider retraining periodically based on new data only. https://docs.aws.amazon.com/machine-learning/latest/dg/retraining-models-on-new-data.html: It is a good practice to continuously monitor the incoming data and retrain your model on newer data if you find that the data distribution has deviated significantly from the original training data distribution
  4. sumanshu 👍 3
    Vote for B. In Recommendation System - Matrix stored in database - which store the users rating and other details like how much time user spent on that page.We generally recommend on basis of matrix. And In production that matrix updated (or model get retrained once a day or once a week) - Because there could be new users rating. or we can say new data. So model retrained fully i.e. on Existing Data + New Data => and generate a new matrix (user/item matrix)
  5. Jai_Deep_Learning 👍 3
    The question says that you know the user preferences keeps changing so new data should also be used. Using existing data is also necessary to understand past behavior. So option B.
  6. sumanshu 👍 3
    Amazon Recommend Method - https://docs.aws.amazon.com/machine-learning/latest/dg/retraining-models-on-new-data.html
  7. rocky48 👍 2 Answered B
    Option A is not recommended because retraining the model on just new data will cause the model to lose the information it has learned from the historical data. Option C and D are not recommended because they are using the new data as test set and this approach will lead to a model that is overfitting and not generalize well to new users. So answer is B
  8. atnafu2020 👍 2
    I thought also C but that's not an answer b/c training and testing is not true with this statement "You know a user's fashion preference is likely to change over time". If it changes all tests in the trained data will fail. it could be B, b/c you cannot/can depend on either of training data only for testing
  9. jvg637 👍 2
    It can be A: Frequently retrain your production models: To capture the evolving and emerging patterns, you need to retrain your model with the most recent data. For example, if your app recommends fashion products using ML, its recommendations should adapt to the latest trends and products.
  10. Arimaverick 👍 2
    B seems to be correct. As new data can be with new features. Hence the new data can be split to both training and test data to retrain as well as with existing data.
  11. jeckie2877 👍 2
    Could be A, but you don't guarantee that will be enough: in practice is B, since you use new and old data( up to some extent) to train your model. So it should be B
  12. rajkinz 👍 2
    Answer is C. It is time sensitive data so latest data should be used for testing. Reference: https://cloud.google.com/automl-tables/docs/prepare#ml-use
  13. samdhimal 👍 2
    Couldn't Agree More. The question seems little vast. It should've specified how a timeframe could play a role as well.
  14. jaimecalderon 👍 2 Answered B
    Continuously retrain the model on a combination of existing data and the new data.
  15. Archy 👍 2
    B ,as until data is not too old, you need to incorporate along with new data.
  16. Ahamada 👍 2 Answered B
    Answer is B, need to take acount both data to train
  17. javibadillo 👍 2 Answered B
    B is the correct answer
  18. rocky48 👍 2
    Nice explanation bro.
  19. Athanasia 👍 2
    answer is D
  20. anton_royce 👍 2
    Answer D
  21. anji007 👍 2
    Ans: B
  22. jin0 👍 1
    there are two point first when retraining second what data. I think retraining should be occur when the model could not predict well in this case there is monitoring metric should be needed first but no one said, second what data? in this case I think the answer is A. because when the model could not predict well it means the data variance and bias are changed so, it's no make sense what is combination new data with old data because the data being not be changed is not necessary anymore..
  23. rtcpost 👍 1 Answered B
    This approach allows the model to benefit from both the historical data (existing data) and the new data, ensuring that it adapts to changing preferences while retaining knowledge from the past. By combining both types of data, the model can learn to make recommendations that are up-to-date and relevant to users' evolving preferences.
  24. cqrm3n 👍 1 Answered B
    We should continuously retrain existing data with latest data to balance the need to adapt to new changes while not overriding historical knowledge. This is called Continuous Retraining where the model is periodically updated with latest data so that recommendations remain accurate over time.
  25. Palanee 👍 1
    Answer B is fair, because the user fashion preference is likely to change overtime and question says we need to build the data pipeline to stream new data back to the model when it is available. So retraining the model with new and existing data would be fair.

Reader comments & corrections

No reader corrections yet — spotted an error or have a note? Share it below.

Log in to comment, report an error, or add a note about this question.

How to establish consistent BigQuery spend?

S-grade

Triggering Airflow DAGs Reactively in Private Cloud Composer

S-grade

How to resolve Cloud Composer 2 worker pod evictions and task failures?

S-grade

Ensuring Consistency in BigQuery Streaming Analytics

S-grade

← Back to PDE Study Guide