Optimizing ML Model Recall for Compliance Risk

PMLE By islever Editorial Last updated

Building models given the task considering cost, complexity, latency, and scalability. Serving models.

Short answer

A Use AutoML to optimize the model’s recall in order to minimize false negatives.

Question

You need to build an ML model for a social media application to predict whether a user’s submitted profile photo meets the requirements. The application will inform the user if the picture meets the requirements. How should you build a model to ensure that the application does not falsely accept a non-compliant picture?
  1. Use AutoML to optimize the model’s recall in order to minimize false negatives. Correct Answer
  2. Use AutoML to optimize the model’s F1 score in order to balance the accuracy of false positives and false negatives.
  3. Use Vertex AI Workbench user-managed notebooks to build a custom model that has three times as many examples of pictures that meet the profile photo requirements.
  4. Use Vertex AI Workbench user-managed notebooks to build a custom model that has three times as many examples of pictures that do not meet the profile photo requirements.

Key Takeaways

In safety-critical ML, the 'bad' outcome is the positive class; minimizing false negatives requires maximizing recall.

Learn why optimizing recall is critical when preventing false negatives in AI compliance tasks. This page explains how to label classes and select metrics to avoid accepting non-compliant content.

Common mistake: Choosing F1 score (B) because it seems balanced, failing to recognize that recall specifically targets the reduction of false negatives, which is the explicit failure mode described.

Expert Analysis

The core of this question lies in defining the 'positive' class for a binary classification problem based on business risk, not just semantic meaning. In machine learning contexts involving safety, fraud, or compliance, the rare or dangerous event is typically defined as the positive class. Here, a non-compliant photo is the negative event that must be caught; therefore, it is labeled as Positive (P). A compliant photo is Negative (N).

The requirement is to ensure the application does not 'falsely accept a non-compliant picture.' If a non-compliant photo (Positive) is accepted, the model has predicted it as Negative. This is a False Negative (FN). To prevent this specific error, we must minimize False Negatives. The metric directly related to minimizing False Negatives is Recall (TP / (TP + FN)). High recall ensures that almost all actual positives (non-compliant photos) are correctly identified.

Option A correctly identifies this by suggesting the optimization of recall. Option B suggests F1 score, which balances precision and recall, but does not prioritize catching the bad case as aggressively as pure recall optimization might. Options C and D involve data manipulation (oversampling) rather than metric optimization, and while oversampling can help with imbalanced data, the primary lever for controlling the type of error (false acceptance vs. false rejection) is the evaluation metric chosen during AutoML tuning. Furthermore, oversampling only one class without addressing the metric can lead to overfitting or biased predictions that don't align with the specific risk tolerance.

Exam Strategy

When an exam question emphasizes preventing a specific type of error (e.g., 'do not falsely accept'), identify that error as either a False Positive or False Negative. If the goal is to catch all instances of a specific condition (the 'positive' class), prioritize Recall. If the goal is to ensure every prediction made is correct, prioritize Precision. Map the business constraint directly to the metric definition before considering data preprocessing steps.

Frequently Asked Questions

Why is the non-compliant photo considered the 'positive' class?

In classification problems focused on risk or anomaly detection, the event of interest or the 'dangerous' outcome is defined as positive. Since we want to detect non-compliant photos, they are the positive class.

What is the difference between optimizing for Recall vs. F1 Score?

Recall focuses solely on capturing all positive instances, minimizing false negatives. F1 Score balances Precision and Recall, which may allow some false negatives if it improves overall harmonic mean balance.

More PMLE FAQ →

Community Votes

B
38%
A
31%
D
31%

38% of anonymous learners picked answer B. Votes are pick records left by other test-takers — they are not the verified answer.

Discussion 69 contributions

Compiled from public community threads and reader submissions, lightly edited for clarity. Handles and vote tallies are shown as recorded, and comments reflect the views of individual contributors.

From public community threads 30 of 69

  1. LearnSodas 👍 21
    I think it's B, since we want to reduce false positives
  2. Anonymous learner 👍 16 Answered A
    The answer is A. The negative event is usually labeled as positive (e.g., fraud detection, customer default prediction, and here non-compliant picture identification). The question explicitly says, "ensure that the application does not falsely accept a non-compliant picture." So we should avoid falsely labeling a non-compliant image as compliant (negative). It is never mentioned in the question that false positives are also a concern. So, recall is better than F1-score for this problem.
  3. taxberg 👍 4 Answered A
    False Negative means it is False that the picture does not follow the requirements -> Accepted. I.e. the model predicts that a non-compliant picture (N) is not against the requirements (F), and so accepted. Tricky one. FP would say that it is false that the picture follows the requirements, and so it rejects a valid image. True Positive + False Negative = Actual Positive = Accepted
  4. neochaotic 👍 4
    D - the question is regarding non-compliant Images. So it makes sense to increase non-compliant examples
  5. hiromi 👍 4 Answered D
    D for me
Show 25 more comments
  1. desertlotus1211 👍 3
    Answer is D: Since the goal is to minimize false positives (incorrectly accepting a non-compliant photo), having more examples of non-compliant photos in the training data will help the model better identify these cases. By training with more non-compliant examples, the model will learn to recognize these images more accurately, thus reducing the chance of falsely accepting them.
  2. lucaluca1982 👍 3 Answered B
    B. Option B, "Use AutoML to optimize the model’s F1 score in order to balance the accuracy of false positives and false negatives", is the best approach to build an ML model that can predict whether a user's submitted profile photo meets the requirements while ensuring that the application does not falsely accept a non-compliant picture.
  3. soumik_barori 👍 3 Answered D
    1. Emphasizes the minority class (non-compliant pictures), ensuring the model better differentiates non-compliant images. 2. Balances the dataset to prevent the model from favouring compliant images disproportionately. 3. Provides flexibility to fine-tune the model for this specific use case.
  4. thescientist 👍 3 Answered D
    D: In this case, a false positive means accepting a non-compliant picture. You want to minimize these. By providing more examples of non-compliant pictures, you train the model to be more sensitive to identifying them and less likely to make this type of error.
  5. pshemol 👍 3 Answered D
    "application does not falsely accept" it's FP not FN so it's not recall but precison. Model must better know wrong photos to make less mistakes on them. Vote for D
  6. niamnesh 👍 3 Answered D
    By oversampling non-compliant photos, you teach the model to better distinguish and not mistakenly accept non-compliant photos, thereby reducing false positives.
  7. jamesking1103 👍 3
    B yes, A is incorrect as minimize false negatives does not help
  8. AB_C 👍 3 Answered D
    D - minimizing false positives
  9. mymy9418 👍 3 Answered A
    optimize recall
  10. YangG 👍 3
    I will go for A
  11. tavva_prudhvi 👍 2
    point is valid. Oversampling the non-compliant class may result in a biased model, especially if the training data does not reflect the true distribution of the classes in the real-world data. This can lead to a model that is more accurate for the oversampled class but performs poorly on the actual data. Moreover, optimizing for recall (i.e., minimizing false negatives) may increase the risk of false positives, which is not desirable for this specific task. It is important to maintain a balance between precision and recall to ensure that the model performs well on both compliant and non-compliant pictures. Therefore, optimizing the model's F1 score, which considers both precision and recall, is a better approach to ensure that the model performs well on both classes while avoiding bias. Additionally, using AutoML can help to select the best model while avoiding bias in the dataset.
  12. tavva_prudhvi 👍 2 Answered B
    In this scenario, it is important to balance the accuracy of false positives (where a non-compliant picture is accepted) and false negatives (where a compliant picture is rejected). By optimizing the F1 score, the model will find the best balance between precision and recall, which will help reduce both false positives and false negatives. This will ensure that the application doesn't falsely accept a non-compliant picture.
  13. pshemol 👍 2
    No it's not tricky. TP - photo meets requirements and it classified as that FP - photo doesn't meet requirements but is classified as that, so it's falsely accepted Your logic makes sense only if you invert the question for classification as "Find photos that don't meet requirements" then FN would mean photo that doesn't meet requirements was classified as it has. (double negation)
  14. desertlotus1211 👍 2
    Optimizing recall reduces false negatives, which is not the primary concern here. The focus is on avoiding false positives—incorrectly classifying non-compliant photos as compliant. While optimizing the F1 score balances false positives and false negatives, it does not prioritize minimizing false positives specifically, which is crucial in this use case.
  15. b1a8fae 👍 2 Answered B
    B. A non-compliant picture is the positive and not the negative. What the question is asking is to decrease the number of false positives ("falsely labeled as non compliant"), which is achieved through optimizing for precision and not recall. Since C and D sound a bit overkill, I would go for the one that prioritizes false positives which is B.
  16. jenjenjenjenjen 👍 2 Answered D
    I think it's D because If you increase the frequency of the negative class in your dataset, it will cause the classifier to become more cautious in its predictions and increase its precision for the negative class. This is because the classifier now has more negative examples to learn from and is less likely to make false positive predictions.
  17. behzadsw 👍 2
    Answer is A (TP/TP+FN). Why do you want to oversample non-compliant? We don't know the distribution in the training data. You might risk a model that is biased.
  18. 8619d79 👍 2 Answered A
    Compliant=negative, accepted non-compliant=false negative (I thought it is negative, to be accepted, but is not). So I need to minimize false negative, Recall
  19. Delphin_8150 👍 2 Answered B
    Gonna go with B on this one, tricky question but since reducing false positives is the goal here only B fits that requirement
  20. moammary 👍 2 Answered A
    Answer is A --> non-compliant photo is positive. Falsely accepting a non-compliant photo is a false negative.
  21. julesnoa 👍 2
    False negative: Non-compliant, but did not alert. That is what we want to minimize.
  22. nimbous 👍 2 Answered D
    oversampling the negative class to avoid falsely labelling them as compliant
  23. enghabeth 👍 2 Answered D
    makes sense to increase non-compliant examples
  24. PhilipKoku 👍 2 Answered A
    A) Minimise False Negatives
  25. Carlose2108 👍 2 Answered A
    I went with A.

Reader comments & corrections

No reader corrections yet — spotted an error or have a note? Share it below.

Log in to comment, report an error, or add a note about this question.

Monitoring ML Model Performance with Precision and Recall Sampling

S-grade

How to Minimize Development Time for Weekly TensorFlow Retraining in Vertex AI?

S-grade

How to Handle Categorical Features in BigQuery ML for Fast Deployment?

S-grade

How to preprocess BigQuery data for TensorFlow?

S-grade

Practice All PMLE Questions

Access 65 questions with complete answers and detailed explanations.

View Full PMLE Practice Test →

← Back to PMLE Study Guide